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Abstract 


New  policy  mandated  by  the  Undersecretary  of  Defense  for  Acquisition, 
Technology  and  Logistics  has  lowered  the  dollar  thresholds  of  contracts  requiring  earned 
value  management  (EVM).  This  policy  directed  that  a  “risk-based  decision”  be  made  to 
determine  if  EVM  should  be  used  on  firm- fixed  price  contracts  under  $20  million. 
Although  not  previously  applicable  to  Military  Construction  (MILCON)  building  projects 
or  other  Air  Force  Civil  Engineer  (CE)  managed  contracts,  the  new  threshold  requires 
that  CE  projects  be  evaluated  for  risk. 

Therefore,  the  focus  of  this  research  was  to  analyze  the  risk  factors  associated 
with  construction  contracts  in  an  attempt  to  build  a  decision  model  to  determine  if  EVM 
is  warranted  on  the  project.  Due  to  the  cost  of  implementing  EVM,  this  model  should 
probably  be  used  only  on  Air  Force  construction  projects  with  cost  estimates  over  $5 
million.  If  the  cost  growth  predicted  by  the  model  is  greater  than  5%,  the  use  of  EVM  is 
recommended  to  monitor  the  risk  factors.  However,  this  recommendation  should  be 
tempered  with  the  overall  risk  associated  with  a  given  project.  In  other  words,  if  the 
calculated  cost  growth  is  high  but  the  probability  of  occurrence  is  low,  the  decision¬ 
maker  may  want  to  forego  the  use  of  EVM  and  the  associated  costs. 
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RISK-BASED  DECISION  MODEL  FOR  DETERMINING  THE  APPLICABILITY  OF 


AN  EARNED  VALUE  MANAGEMENT  SYSTEM  IN  CONSTRUCTION 


CHAPTER  1 
INTRODUCTION 


Construction  is  not  an  exact  science.  Unlike  a  mathematical  equation,  there  are 
factors  associated  with  construction  that  are  out  of  the  control  of  the  project  manager  but 
greatly  affect  the  success  of  the  project.  In  the  construction  business,  success  is  typically 
measured  by  three  basic  performance  criteria  often  called  the  triple  constraint:  cost, 
schedule,  and  quality.  However,  inherent  in  all  construction  projects  are  factors  that 
influence  these  criteria  and  ultimately  the  success  of  the  project.  These  factors  are  the 
many  uncertainties  associated  with  construction;  they  represent  risk  whenever  a  building 
is  being  built  or  renovated.  There  is  risk  to  the  owner  due  to  their  investment  of  capital; 
there  is  also  risk  to  the  construction  contractor  due  to  safety  liability,  failure  of  a  sub¬ 
contractor,  or  owner  default  (Mustafa  and  Al-Bahar,  1991).  A  project  manager  must  try 
to  anticipate  these  risks  in  order  to  effectively  mitigate  their  impacts.  Without 
anticipation,  the  project  manager  is  forced  to  put  out  the  proverbial  fires  as  they  arise; 
however,  it  is  usually  better  to  be  proactive  than  reactive.  This  is  the  challenge  any 
project  manager  faces. 
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Background 

The  heart  of  project  management  is  the  effective  management  of  the  competing 
interests  represented  by  the  triple  constraint.  Assuming  quality  is  a  direct  result  of  the 
design,  the  project  manager  (PM)  focuses  primarily  on  the  cost  and  schedule  factors 
using  a  variety  of  tools.  These  tools  are  briefly  discussed  in  the  following  sections  before 
introducing  the  concept  of  earned  value  management. 

Construction  Management 

A  project’s  success  relies  on  three  facets:  design,  cost  and  schedule. 

Construction  management  becomes  involved  after  the  design  is  complete.  Therefore, 
management  is  typically  concerned  with  a  project’s  adherence  to  the  budgeted  cost  and 
the  proposed  schedule.  To  control  costs  and  schedule,  many  techniques  are  available  to 
the  project  manager.  Cost  can  be  a  function  of  a  Work  Breakdown  Structure  (WBS) 
using  either  a  Schedule  of  Values  or  the  Critical  Path  Method  (CPM)  (Fleming  and 
Koppelman,  2002).  The  schedule  can  be  shown  through  the  use  of  a  Gantt  Chart  in  either 
the  Program  Evaluation  and  Review  Technique  (PERT)  or  using  the  CPM  (Mantel  et  ah, 
2005).  These  methods  are  techniques  used  by  project  managers  to  ensure  project  success. 
However,  Earned  Value  Management  provides  the  project  manager  the  most  effective 
tool  through  the  integration  of  cost  and  schedule  (Christensen,  1998). 

Earned  Value  Management 

The  tools  discussed  above  are  extremely  valuable  to  the  PM;  however,  they  look 
at  cost  and  schedule  as  separate  entities.  Since  cost  and  schedule  are  related,  the  concept 
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of  earned  value  management  (EVM)  is  becoming  more  widely  used.  EVM  is  a  tool 
which  helps  the  PM  determine  “the  accurate  measurement  of  physical  performance 
against  a  detailed  plan  to  allow  for  the  accurate  prediction  of  the  final  costs  and  schedule 
results  for  a  given  project”  (Fleming  and  Koppelman,  2000).  The  EVM  system  has  its 
roots  in  the  Department  of  Defense  (DoD),  where  it  was  used  as  early  as  the  1960s 
(Christensen,  1998).  Its  genesis  was  the  Air  Force’s  establishment  of  cost/schedule 
planning  and  control  specifications  (C/SPEC)  to  set  a  standard,  provide  consistency,  and 
properly  manage  the  project  management  and  administration  system.  The  other  military 
services  implemented  the  concept  and  it  became  known  as  Cost/Schedule  Control 
Systems  Criteria  (C/SCSC).  It  was  not  until  the  1990s  that  industry  adopted  the  concept, 
renamed  it  Earned  Value  Management  System  (EVMS),  and  established  a  formal 
process.  The  C/SCSC  contained  35  management  criteria  that  a  contractor  must  meet  to 
bid  on  government  contracts.  These  35  DoD  criteria  were  then  consolidated  by  industry 
into  32  criteria  organized  in  five  sections:  Organization,  Panning  and  Budgeting, 
Accounting,  Analysis,  and  Revisions. 

DoD  Use  of  EVM 

The  use  of  EVM  in  the  DoD  is  ubiquitous  in  the  acquisition  communities  of  all 
services.  In  fact,  acquisition  policy  has  established  a  threshold  of  $3 15  million,  above 
which  EVM  is  mandated  in  contract  requirements.  The  result  is  that  many  major  weapon 
system  contractors  have  become  very  familiar  with  its  use.  Fleming  and  Koppelman 
(2002)  argue  that  EVM  is  just  as  useful  and  just  as  effective  at  mitigating  risk  on 
construction  projects  as  it  has  proven  to  be  on  large  weapon  system  contracts.  To 
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increase  the  use  of  EVM  throughout  the  DoD,  a  new  policy,  dated  7  March  2005, 
requires  all  contracts  valued  over  $20  million  to  use  EVM.  Furthermore,  it  also  states 
that  all  contracts  over  $50  million  must  require  the  contractor’s  EVM  system  to  be 
validated.  For  contracts  less  than  $20  million,  the  policy  states,  “While  not  required,  if  a 
risk-based  decision  is  made  to  require  EVM. . .”  it  should  be  a  contract  requirement. 
Furthermore,  it  also  states  that  all  contracts  over  $50  million  must  require  the  contractor’s 
EVM  system  to  be  validated.  For  contracts  less  than  $20  million,  the  policy  states, 
“While  not  required,  if  a  risk-based  decision  is  made  to  require  EVM. . .”  it  should  be  a 
contract  requirement.  Presumably,  the  rationale  for  the  risk-based  approach  is  the 
implementation  cost  associated  with  EVM,  which  Christensen  (1998)  estimates  ranges 
from  0.1%  up  to  5%  of  the  total  project  cost.  The  upper  limit  is  typically  equal  to  the 
entire  contingency  fund  on  a  construction  project;  therefore,  EVM’s  use  is  not 
appropriate  for  all  projects.  However,  no  further  explanations  or  criteria  are  provided  to 
establish  which  risks  should  be  considered.  Therefore,  the  risk-based  decision  levied  by 
the  policy  letter  potentially  impacts  all  contract  management  personnel. 

Risk 

Project  risk  can  be  defined  as  the  cumulative  effect  of  the  chances  of  uncertain 
occurrences  adversely  affecting  project  objectives  (Wideman,  1992).  Unfortunately, 
project  risk  is  always  present  and  unknowns  are  unavoidable.  To  be  successful  though,  it 
is  incumbent  on  the  project  manager  to  attempt  to  anticipate  and  mitigate  these  risks.  The 
key  is  to  understand  the  possible  risks  associated  with  a  construction  project  and 
determine  if  a  particular  project  has  sufficient  inherent  risk  to  warrant  EVM.  In  this  way, 
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the  project  manager  can  institute  procedures  to  mitigate  the  impact  of  a  risk  event  when  it 
occurs.  Fleming  and  Koppehnan  (2002)  may  have  summed  it  up  best  when  they  asserted 
that: 

The  basic  utility  of  [an  aggressive  project  management  technique]  is  to 
contain  the  cost  risks  associated  with  projects.  Bad  news  never  gets  better 
with  time.  The  earlier  you  know  that  you  have  a  problem  on  your  project, 
the  better  chance  you  will  have  to  mitigate  that  problem. 

Given  this  brief  introduction  to  risk,  the  problems  and  risk  factors  associated  with 

construction  can  be  addressed. 

Although  every  construction  project  is  different  and  contains  its  own  risk  factors, 
cost  and  schedule  delays  are  typically  a  result  of  five  overarching  factors:  physical 
attributes,  design  specifications,  contractor,  designer,  and  owner  (Jeljeli  and  Russell, 
1995).  Physical  attributes  include  differing  site  conditions,  weather,  technology,  etc. 
Design  specifications  may  include  constructability  issues,  scope  changes,  accuracy  of  the 
design,  and  age  of  the  design.  The  risks  associated  with  the  contractor  are  related  to 
experience,  past  perfonnance,  and  management.  A  designer  or  architect  similarly  brings 
risks  due  to  experience,  past  performance,  and  proximity  to  the  site.  The  reference  to  the 
owner  is  for  lack  of  funds,  which  is  typically  not  as  much  of  a  factor  for  government 
contracts. 

Should  unforeseen  events  occur,  the  PM  must  effectively  manage  these  risks  in 
order  to  limit  the  impact  to  the  schedule  and/or  cost.  To  do  this,  the  project  manager  has 
a  contingency  set-aside;  however,  this  amount  is  arbitrarily  set  and  generally  calculated 
as  a  small  percentage  of  the  total  project  cost.  The  contingency  fund  allows  the  PM  to 
avoid  cost  overruns  to  a  certain  point,  but  changes  inevitably  result  in  schedule  delays. 
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Thus,  the  ability  to  integrate  cost  and  schedule  into  one  measure  is  the  objective  of  earned 
value. 

All  projects  have  risk  -  to  attempt  a  risk  free  project  would  not  only  be  impossible 

but  too  costly  and  time  consuming  to  implement  appropriate  risk  avoidance  techniques 

(Kendrick,  2003).  So  why  do  risk  analysis?  Kendrick  (2003)  puts  it  very  succinctly: 

Project  risk  management  depends  on  thorough,  sustained  application  of 
effective  project  management  principals.  The  precise  nature  of  the  project 
management  methodology  can  vary  widely,  but  management  of  risk  is 
most  successful  when  consistent  processes  are  adopted  by  the  organization 
as  a  whole,  because  there  is  more  information  to  work  with  and  more 
durable  support  for  the  ongoing  effort  required. 

It  is  this  management  of  risk  that  comes  full  circle  back  to  the  DoD’s  policy  regarding  a 

“risk-based  decision”  and  the  use  of  EVM.  Therefore,  it  is  imperative  that  a  quantitative 

method  be  developed  to  accurately  assess  the  specific  risks  associated  with  construction 

projects. 

Air  Force  Concerns 

Within  the  DoD,  the  Air  Force  (AF)  experiences  similar  problems  related  to 
managing  risks  and  minimizing  the  impact  on  cost  and  schedule  delays.  For  instance,  the 
proposed  Air  Force  military  construction  (MIFCON)  budget  in  fiscal  year  (FY)  2006  is 
$4.7  billion.  Since  the  AF’s  goal  is  to  manage  construction  cost  growth  to  5%  or  less  of 
the  original  contract  amount,  the  contingency  fund  reserved  for  MIFCON  projects  could 
amount  to  $235  million.  Therefore,  it  is  imperative  that  AF  construction  project 
managers  anticipate  risk  factors  and  implement  aggressive  project  management 
techniques  to  mitigate  these  factors.  Currently,  the  AF  relies  on  the  Army  Corps  of 
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Engineers  to  manage  MILCON  projects;  however,  the  AF  project  manager  must  still  deal 
with  the  budgetary  constraints  and  the  impact  of  cost  overruns  and  schedule  delays. 

Problem  Statement 

The  policy  letter  and  the  associated  rationale  reference  the  need  to  perfonn  a 
“risk-based”  assessment  in  order  to  determine  if  EVM  is  warranted  on  contracts  less  than 
$20  million.  Therefore,  this  research  will  attempt  to  help  the  project  manager  by  building 
a  decision  model  to  evaluate  the  risks  associated  with  military  construction  projects  and 
provide  a  quantitative  analysis  to  detennine  whether  EVM  should  be  used  on  a  particular 
construction  project.  Therefore,  the  question  that  arises  is  how  to  assess  the  risks 
associated  with  AF  construction  projects  in  order  to  determine  if  sufficient  uncertainty 
exists  to  warrant  EVM. 

Research  Objective 

The  objective  of  the  research  is  to  develop  a  model,  which  evaluates  risk  factors 
associated  with  construction,  the  risk’s  associated  probability  of  occurrence,  and  the 
subsequent  cost  growth  impact.  The  model  will  yield  an  expected  cost  growth,  which 
will  then  be  used  to  determine  whether  EVM  should  be  recommended.  The  ensuing 
investigative  questions  addressed  during  this  research  are  listed  below. 

1 .  What  risk  factors  affect  the  cost  or  schedule  of  Air  Force  construction 
projects? 

2.  What  is  the  probability  of  occurrence  of  the  identified  risk  factors? 

3.  What  is  the  subsequent  cost  growth  due  to  the  occurrence  of  the  risk  factor? 
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4.  How  many  Air  Force  construction  projects  accomplished  in  the  last  5  years 
and  those  programmed  through  the  FY12  exceed  the  $20M  threshold  or  fall 
into  a  range  which  must  be  evaluated? 

Methodology 

The  following  decision  analysis  methods  were  used  to  quantify  the  risk, 
probability,  and  cost  growth  associated  with  construction:  Alternative-Focused  Thinking 
(AFT),  Value-Focused  Thinking  (VFT),  Influence  Diagrams,  and  Decision  Trees. 
Although  AFT  and  VFT  are  excellent  at  breaking  down  a  decision  into  its  component 
elements  and  evaluating  alternatives  based  on  a  decision  maker’s  preferences,  the 
limitation  is  that  the  resulting  values  are  not  scalable  to  each  other.  In  other  words,  there 
would  be  no  way  to  evaluate  a  construction  project’s  risk  and  compare  it  to  an  established 
project  that  warrants  EVM.  Influence  diagrams  are  very  useful  because  they  establish  a 
visual  representation  of  the  interaction  between  factors  affecting  a  decision  but  fall  short 
in  the  analysis  of  the  “chronological  structure  and  state  of  uncertain  events”  (Jeljeli  and 
Russell,  1995). 

The  decision  tree  model  allows  for  the  exploration  of  as  many  of  the  risk  factors 
as  possible.  The  decision  tree  provides  the  breakdown  of  the  component  elements  of  a 
decision,  integrates  uncertainty,  and  yields  an  outcome  which  optimizes  the  expected 
value  (Clemen  and  Reilly,  2001).  The  model  will  attempt  to  anticipate  ah  possible 
outcomes  resulting  from  the  unknowns  related  to  any  construction  project,  such  as 
weather  delays,  unforeseen  sight  conditions,  problems  related  to  renovation  versus  new 
construction,  pitfalls  of  a  design-bid-build  contract  versus  a  design-built  type  of  contract. 
The  decision  tree  will  in  this  respect  explore  ah  possible  risk  scenarios  to  yield  a  factor  of 
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growth  of  the  project  cost.  Based  on  this  cost  growth  the  decision  maker  will  have 
additional  insight  regarding  the  appropriateness  of  using  EVM  to  manage  the  project  risk. 

Assumptions  and  limitations 

Several  assumptions  and  limitations  are  relevant  for  this  research.  The  primary 
assumption  is  that  the  revised  DoD  EVM  policy  letter  dated  7  March  2005  is  applicable 
to  Air  Force  construction  contracts.  Additionally,  this  research  assumes  that  the  risk- 
based  decision  called  for  in  the  revised  policy  letter  requires  more  than  a  subjective 
evaluation  by  a  decision  maker.  The  primary  limitation  affecting  this  research  is  with 
respect  to  the  accuracy  of  the  data  used  to  build  the  model.  Although  cost  growth  data  is 
contained  in  the  Automated  Civil  Engineer  System  (ACES),  this  thesis  will  focus  on  the 
use  of  expert  opinion  to  provide  quantifiable  data.  A  limitation  associated  with  expert 
opinion  is  that  it  may  be  subject  to  biases  and  influences  not  easily  controlled.  Therefore, 
the  results  will  be  subject  and  open  for  interpretation  and  disagreement  but  the  data 
acquired  is  assumed  to  be  true  and  accurate. 

Significance  of  Study 

The  requirement  to  assess  the  risks  associated  with  construction  projects  because 
of  the  revised  DoD  EVM  policy  is  new.  The  standard  practice  is  to  assign  a  5% 
contingency  amount  to  construction  projects  to  cover  unforeseen  events.  The  revised 
policy  requires  that  a  decision  be  made  based  on  the  risk  in  order  to  detennine  the 
applicability  of  EVM  to  the  project.  The  significance  of  this  research  is  that  it  will 
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provide  a  quantifiable  process  for  determining  the  level  of  risk  associated  with  a  specific 
construction  project  to  make  an  infonned  decision  about  the  applicability  of  EVM. 

Organization  of  Remaining  Chapters 

The  subsequent  chapters  will  cover  the  relevant  literature,  the  methodology,  the 
results  and  finally  the  conclusion  of  this  research.  Chapter  2  details  the  literature 
applicable  to  construction  management,  earned  value  management,  risk,  and  decision 
analysis.  Chapter  3  covers  the  methodology  used  in  this  research;  it  focuses  on  the 
Delphi  method  and  the  use  of  decision  trees.  The  results  will  be  analyzed  and  explained 
in  Chapter  4.  The  final  chapter  will  discuss  the  conclusions  reached  from  this  overall 
research  effort. 
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CHAPTER  2 


LITERATURE  REVIEW 


Chapter  2  provides  detail  into  the  relevant  subject  areas  that  this  research  utilizes. 
The  recent  literature  in  the  areas  of  construction  management,  earned  value  management, 
decision  analysis,  and  risk  analysis  is  reviewed.  The  first  section  discusses  construction 
management  and  specifically  focuses  on  the  techniques  to  monitor  a  project’s  cost  and 
schedule.  A  specialized  method  of  tracking  project  cost  and  schedule,  Earned  Value 
Management  (EVM),  is  covered  as  well.  Decision  Analysis  (DA)  techniques  such  as 
Alternative-Focused  Thinking  (AFT),  Value-Focused  Thinking  (VFT),  Influence 
Diagrams,  and  Decision  Trees  are  then  discussed.  Finally,  the  concept  of  risk  is 
discussed  in  terms  of  its  fundamentals,  management,  and  existing  research  in 
construction  risk  assessment. 

Construction  Management 

All  project  managers  (PMs)  deal  with  the  three  factors  affecting  any  construction 
effort:  design,  cost,  and  schedule.  Often,  the  design  is  already  established;  therefore,  the 
only  control  over  the  project’s  success  or  failure  is  the  effective  management,  oversight, 
and  control  of  the  project’s  cost  and  schedule.  The  original  cost  estimates  are  typically 
used  to  develop  the  project  budget,  which  then  serves  as  a  road  map  for  the  expenditure 
of  money  related  to  each  phase  of  the  construction.  The  accompanying  schedule  is 
generally  based  on  either  time  or  resource  constraints.  Time-based  scheduling  refers  to  a 
critical  path  method  in  which  completion  of  one  phase  is  required  before  the  next  phase 
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can  begin.  Resource-based  scheduling  is  based  on  the  necessity  of  manpower, 
equipment,  and  materials  to  accomplish  the  construction  (Hendrickson,  2003).  The  PM 
has  multiple  tools  available  to  monitor  cost  and  schedule. 

Monitoring  Cost 

Since  Hendrickson  (2003)  states  that  the  “original  detailed  cost  estimate  is 
typically  converted  to  a  project  budget ,”  the  budget  is  often  considered  the  primary  tool 
to  monitor  cost.  Using  the  delineation  of  work  represented  by  the  detailed  cost  estimate, 
each  element  of  work  thus  has  an  associated  cost  and  is  typically  assigned  to  a  cost 
account.  These  cost  accounts  begin  at  major  division  levels  such  as  site  preparation; 
foundations;  heating,  ventilation,  and  air  conditioning  (HVAC);  electrical;  mechanical 
and  so  forth  with  sub-divisions  which  break  down  each  major  division  into  the  smallest 
possible  element  that  can  be  tracked  (Hendrickson,  2003).  The  final  product  is  a  Work 
Breakdown  Structure  (WBS)  that  represents  all  tasks  for  a  given  project.  The  WBS  is 
typically  shown  graphically  as  a  tree  with  each  branch  delineating  major  divisions  and 
expanding  further  to  show  work  elements  (Mantel  et  al.,  2005).  Since  the  WBS  is 
strongly  related  to  the  cost  estimate,  tracking  expenses  versus  the  planned  work  allows 
for  an  objective  assessment  of  cost  accounting.  Figure  2. 1  shows  a  WBS  from  the  work 
breakdown  and  the  cost  accounting  view. 
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As  work  progresses,  the  contractor  submits  invoices  that  reflect  the  materials  and 
labor  expended  to  accomplish  the  individual  work  element.  However,  complicating 
factors  are  inevitable;  as  the  project  experiences  scope  changes,  unforeseen  sight 
conditions,  weather  delays,  or  any  number  of  other  complications,  actual  costs  will 
deviate  from  the  budget.  Therefore,  the  PM  must  incorporate  cost  control  tools  to 
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adequately  track  the  variance  between  actual  cost  outlays  and  the  planned  budget.  Two 
methods  of  controlling  costs  are  the  Schedule  of  Values  and  Critical  Path  Method  (CPM). 
The  Schedule  of  Values  is  a  line-by-line  accounting  of  the  work  to  be  performed  and  the 
costs  associated  with  the  work.  As  the  project  progresses,  the  PM  can  itemize  the  line 
items  being  billed  and  track  the  work  versus  the  project  budget  (Fleming  and  Koppehnan, 
2002).  Although  CPM  is  primarily  used  for  scheduling,  it  can  also  be  used  for  costs  if 
the  specific  costs  are  attributed  to  the  individual  work  packages  shown  in  the  CPM 
(Fleming  and  Koppehnan,  2002). 

The  PM  can  also  use  the  cost  accounts  from  the  WBS  to  effectively  manage  cost 
variances.  For  example,  the  labor  required  to  install  the  electrical  outlets  in  a  room  on  a 
particular  floor  could  be  the  lowest  sub-division  on  a  WBS.  Therefore,  if  twice  as  much 
labor  is  required  to  perfonn  the  work  than  initially  budgeted,  the  PM  is  able  to  track  the 
incremental  cost  growth  of  the  labor.  In  this  way,  the  PM  is  able  to  monitor  cost  but  has 
little  control  over  preventing  such  occurrences.  Similarly,  if  the  work  takes  twice  as  long 
to  perform  than  originally  estimated,  the  PM  is  able  to  track  the  impact  on  the  project 
schedule. 

Monitoring  Schedule 

The  primary  tool  in  schedule  management  is  the  project  baseline  (Fleming  and 
Koppehnan,  2002).  The  first  attempt  to  quantify  the  project  was  the  Gantt  chart,  which 
was  developed  in  1917  in  an  effort  to  identify  graphically  all  the  steps  required  to 
complete  a  project.  The  benefits  of  the  chart  are  that  the  work  is  broken  down  into 
specific  work  elements  and  the  entire  project  is  shown  as  a  progressive  iteration  of  work 
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packages.  The  limitation,  however,  is  when  multiple  tasks  have  concurrent  start  times 
and  similar  durations.  The  overlapping  start  times  and  finish  times  reduce  the 
effectiveness  of  the  technique  because  the  sequence  of  tasks  that  drive  the  schedule  are 
indistinguishable  in  the  chart  from  those  tasks  which  are  independent  of  the  critical  path 
(Mantel  et  ah,  2005). 

Based  on  the  Gantt  chart,  the  most  common  standard  formats  for  monitoring  the 
project  baseline  are  the  Program  Evaluation  and  Review  Technique  (PERT)  and  the 
Critical  Path  Method  (CPM).  These  techniques  were  developed  independently  in  the 
1950s  and  depend  on  the  order  and  sequence  of  the  work  to  be  accomplished.  Each 
method  tracks  task  completion,  which  is  considered  to  be  dependent  and/or  contingent  on 
the  success  of  a  previous  task  or  a  set  of  tasks.  For  example,  if  a  task  is  not  completed  on 
time,  it  creates  delays  in  succeeding  tasks  and  the  entire  project  if  it  is  on  the  critical  path 
(Mantel  et  ah,  2005). 

Although  the  distinction  between  PERT  and  CPM  is  trivial  in  execution  when 
managing  a  project  baseline  schedule,  the  methods  differ  greatly.  PERT  employs 
probabilistic  estimates  based  on  the  uncertainty  associated  with  task  durations,  while 
CPM  assumes  100%  certainty  in  time  estimates  to  develop  deterministic  task  durations. 
Despite  the  differences  in  methodology,  the  symbolic  nature  of  the  PERT  and  CPM 
techniques  eliminates  the  ambiguity  of  concurrent  tasks  in  the  Gantt  chart  by  focusing  on 
the  technical  details.  These  methods  integrate  the  graphical  nature  of  the  Gantt  chart  with 
the  technical  details  pertaining  to  each  activity.  The  details  included  are  activity  name, 
activity  duration,  earliest  start  time  (EST),  earliest  finish  time  (EFT),  latest  start  time 
(LST),  latest  finish  time  (LFT),  and  slack  (Mantel  et  ah,  2005). 
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Regardless  of  the  method  chosen,  the  project  manager  uses  these  techniques  only 
to  monitor  schedule  progress  and  not  to  affect  change.  To  affect  change,  critical 
knowledge  of  a  project  must  be  obtained  regarding  the  combined  effect  of  both  cost  and 
schedule  progress.  The  analysis  of  the  link  between  cost  and  schedule,  often  referred  to 
as  earned  value  management,  provides  the  most  powerful  tool  to  control  future  cost  and 
schedule  growth  (Christensen,  1998). 

Earned  Value  Management 

Earned  value  management  (EVM)  is  a  management  tool  that  allows  for  the 
integration  of  a  performance  schedule  with  a  monetary  outlay,  or  payment  schedule.  To 
gain  a  better  understanding  of  EVM,  the  rest  of  this  section  discusses  its  history  and  key 
components.  Additionally,  the  cost  of  implementing  EVM  and  its  applicability  to  the 
construction  field  are  discussed. 

History 

The  concept  of  EVM  has  its  roots  in  the  Department  of  Defense  (DoD).  As  early 
as  the  1960s,  the  DoD  was  using  the  concept  of  earned  value  as  a  way  to  compare 
planned  work  versus  completed  work  (Christensen,  1998).  The  Cost/Schedule  Planning 
and  Control  Specifications  (C/SPEC)  Group  was  created  by  the  Air  Force  to  generate 
standards  and  measures  by  which  contractors  were  evaluated  before  being  awarded  large 
contracts  (Smith,  2005).  To  properly  manage  the  concept,  specifications  were  identified 
to  quantify  the  standard  measurement  among  companies  who  wished  to  bid  on 
government  contracts.  These  specifications  laid  out  a  systematic  management  control 
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function  that  contractors  were  required  to  have,  use,  and  maintain  to  update  project  cost 
and  schedule  progress.  The  C/SPEC  also  provided  consistency  in  the  calculation  and 
evaluation  of  cost  and  schedule  factors,  equations,  and  results,  which  form  the  basis  of 
EVM.  Ultimately,  these  specifications  evolved  in  to  the  Cost/Schedule  Control  Systems 
Criteria  (C/SCSC),  which  subsequently  became  the  standard  within  DoD  (Christensen, 
1998). 

It  was  not  until  the  1990s  that  industry  adopted  the  concept,  renamed  it  the  earned 
value  management  system  (EVMS),  and  established  a  formal  earned  value  process. 
Industry  reorganized  the  previous  DoD  criteria  and  categorized  32  criteria  into  five 
sections:  Organization,  Panning  and  Budgeting,  Accounting,  Analysis  and  Management 
Reporting,  and  Revisions  and  Data  Maintenance.  The  DoD  quickly  implemented  these 
revised  criteria  as  the  new  government  standard.  These  criteria  were  subsequently 
adopted  in  1998  as  the  American  National  Standards  Institute/Electronic  Industries 
Alliance  Standard  748,  Earned  Value  Management  Systems  (ANSI/EIA-748).  Thus,  in 
almost  50  years,  EVM  has  developed  from  a  concept  used  primarily  in  the  military  to  an 
industry  standard. 

Components 

To  track  a  project’s  progress  against  the  baseline  plan,  three  key  components  must 
be  understood:  Planned  Value,  Earned  Value,  and  Actual  Costs.  Planned  Value  is  the 
cost  estimate  at  the  beginning  of  the  project  based  on  the  project  baseline,  otherwise 
known  as  Budgeted  Cost  of  Work  Scheduled  (BCWS).  The  second  factor  is  Earned 
Value,  which  is  tracked  during  execution  of  the  project  and  represents  the  cost  of  the 
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work  that  has  been  authorized  and  completed  (also  known  as  Budgeted  Cost  of  Work 
Performed  (BCWP)).  The  third  factor,  Actual  Costs,  represents  the  “actual  costs  incurred 
to  convert  the  Planned  Value  into  the  Earned  Value”  and  is  typically  known  as  the  Actual 
Cost  of  Work  Performed  (ACWP)  (Fleming  and  Koppehnan,  2002).  Using  these  factors, 
a  project  manager  is  able  to  calculate  several  other  parameters  and  indices  of  interest, 
which  help  provide  a  better  understanding  of  a  project’s  perfonnance. 

Project  Performance 

The  Performance  Measurement  Baseline  (PMB)  integrates  the  project  baseline 
and  the  cost  associated  with  each  element;  it  is  the  standard  by  which  the  project  is 
compared  from  start  to  finish.  The  difference  between  the  PMB  and  the  actual  cost 
expenditures  updated  continuously  throughout  the  project  is  measured  two  ways.  The 
first  is  related  to  cost  and  the  second  is  associated  with  schedule.  Cost  Variance  (CV)  is 
calculated  by  subtracting  the  ACWP  from  the  BCWP.  The  result  indicates  whether  the 
project  is  costing  more  or  less  than  the  PMB  at  a  given  time  during  the  project.  A 
negative  value  indicates  that  the  project  is  over  budget,  while  a  positive  result  means  the 
project  is  under  budget  (Mantel  et  ah,  2005).  Schedule  Variance  (SV)  is  calculated  by 
subtracting  the  BCWS  from  the  BCWP.  The  result  indicates  whether  the  project 
schedule  is  ahead  of  or  behind  the  PMB.  A  negative  value  indicates  that  the  project  is 
behind  schedule,  while  a  positive  result  means  the  project  is  ahead  of  schedule  (Mantel  et 
ah,  2005). 

In  addition  to  these  parameters,  perfonnance  indices  are  used  to  determine  the 
status  of  a  project  and  calculate  the  expected  project  completion  date  and  final  cost 
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(Stockman,  2006).  These  performance  indices  are  comprised  of  three  factors:  Cost 
Performance  Index  (CPI),  Schedule  Performance  Index  (SPI),  and  Cost  Schedule  Index 
(CSI).  The  CPI,  calculated  by  dividing  the  BCWP  by  the  ACWP,  represents  the  ratio  of 
work  accomplished  versus  the  cost  to  accomplish  the  work;  it  is  often  called  a  cost 
efficiency  rate.  If  the  ratio  is  greater  than  one,  it  indicates  that  more  than  a  dollar’s  worth 
of  work  was  accomplished  for  every  dollar  spent.  A  ratio  less  than  one  shows  that  it  is 
costing  more  to  do  the  work  than  the  baseline  projected.  In  a  study  of  over  700  DoD 
projects,  the  CPI  was  found  to  stabilize  by  the  time  a  project  was  20%  complete.  In  fact, 
the  study  indicated  that  the  “percent  overrun  at  completion  will  be  within  10%  of  percent 
overrun  at  20%  complete”  (Fleming  and  Koppelman,  2002).  This  means  that  a  final 
project  cost  can  be  forecasted  within  10%  as  early  as  20%  into  the  project.  Similar  to  the 
CPI,  the  SPI  is  calculated  by  dividing  the  BCWP  by  the  BCWS.  It  represents  the  ratio  of 
work  accomplished  versus  the  value  of  the  planned  work.  If  the  ratio  is  greater  than  one, 
it  indicates  that  the  cost  of  work  completed  is  greater  than  the  planned  amount;  a  ratio 
less  than  one  shows  that  the  cost  of  work  completed  is  less  then  the  planned  amount.  The 
third  factor,  CSI,  is  calculated  by  multiplying  the  CSI  and  the  SPI.  The  results  are  a 
combined  index  of  the  cost  and  schedule  performance. 

Once  these  indices  are  calculated,  additional  information  about  the  project  can  be 
calculated.  The  project  completion  estimates  are  Estimate  to  Complete  (ETC),  Estimate 
at  Completion  (EAC),  and  Variance  at  Completion  (VAC).  The  ETC  is  the  difference 
between  the  Budget  at  Completion  (BAC)  and  the  BCWP  divided  by  the  CPI.  It  applies 
the  observed  cost  efficiency  to  estimate  the  cost  to  finish  the  remaining  work.  The  EAC 
is  simply  the  ETC  plus  the  ACWP,  and  the  VAC  is  the  difference  between  the  BAC  and 
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the  EAC.  The  VAC  shows  how  much  over  (or  under)  budget  the  project  is  estimated  to 
cost.  Figure  2.2  graphically  shows  the  performance  indices  and  their  relationship  to  the 
original  estimate,  which  is  the  PMB. 


Cost 


Time  Scale 


Figure  2.2  Earned  Value  Performance  Indices  (Stockman,  2006) 
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Cost  of  EVM 


The  benefit  and  power  of  EVM  does  not  come  without  costs.  In  an  analysis  of  six 
studies  done  on  the  cost  of  EVM  across  industry  and  the  DoD,  it  was  found  that  the  cost 
of  EVM  could  range  from  as  little  as  0.1%  up  to  5%  of  the  total  project  cost  (Christensen, 
1998).  Therefore,  EVM  has  traditionally  been  used  on  large  cost-reimbursable  contracts 
for  major  DOD  weapon  system  development  and  purchase  where  the  cost  of  EVM  is  far 
exceeded  by  the  benefits  attributed  to  project  management  and  oversight. 

Construction  Applicability 

The  upper  limit  of  EVM  cost  (i.e.,  5%)  is  equal  to  the  entire  contingency  budget 
of  a  typical  Air  Force  construction  project;  therefore,  EVM  has  not  been  used  in  the 
construction  field.  However,  Fleming  and  Koppehnan  (2002)  contend  that  EVM  is 
warranted  for  construction  projects  and  propose  six  recommendations. 

•  Require  the  construction  contractor  to  provide  a  “time-phased  Schedule  of  Values 
in  which  the  sum  of  the  line  items  will  add  up  to  the  total  contract  value.”  This 
can  be  thought  of  as  the  Planned  Value  or  BCWP.  In  terms  of  construction,  this 
resembles  a  Gantt  chart  in  which  the  costs  associated  with  each  work  package  are 
integrated  with  the  chart. 

•  Require  the  construction  contractor  to  provide  monthly  invoices  of  actual  costs 
incurred  and  to  update  the  Schedule  of  Values  with  current  progress.  This 
provides  the  BCWP  and  ACWP  values  necessary  to  calculate  cost  and  schedule 
variance. 
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•  Calculate  and  compare  the  Schedule  Perfonnance  Index  (SPI)  and  the  Cost 
Perfonnance  Index  (CPI)  not  only  as  a  gauge  for  an  individual  project  but  also  as 
a  means  to  compare  projects  from  an  enterprise  view. 

•  Calculate  the  estimate  for  final  cost  and  detennine  if  that  value  exceeds  the 
owner’s  risk. 

•  Require  suppliers  of  the  contractor  to  provide  a  “time-phased  schedule  of  value.” 
Shown  graphically,  this  should  resemble  an  S-shaped  curve  to  reflect  a  gradual 
build-up  of  inventory,  transitioning  to  a  more  rapid  build-up,  and  then  a  slow 
decline  at  the  end  of  a  project.  Something  other  than  an  S-shaped  curve  would  be 
a  warning  sign. 

•  Require  the  contractor  to  track  payments  to  suppliers.  This  recommendation  is 
designed  to  monitor  and  avoid  contractors  experiencing  loss  due  to  a  differential 
between  what  has  been  forecasted  and  what  is  actually  paid  to  suppliers. 

The  effect  of  these  recommendations  is  the  mitigation  of  risk,  which  should  be  a  goal  of 
all  project  managers.  All  construction  has  risk,  but  the  ability  to  forecast  or  anticipate 
risk  prevents  excessive  cost  and  schedule  overruns  and  thus  provides  for  a  more 
successful  project. 

Earned  Value  Management  in  the  DoD 

Prior  to  March  2005,  DoD  directives  established  two  thresholds  for  using  EVM. 
The  first  applied  to  contracts  related  to  Research,  Development,  Test,  and  Evaluation 
(RTD&E)  and  required  EVM  on  projects  greater  than  $73  million.  The  second  applied  to 
Procurement/Operations  and  Maintenance  (O&M),  which  includes  construction  projects, 
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and  required  EVM  on  contracts  greater  than  $315  million.  In  March  2005,  the  DoD 
revised  its  policy  regarding  the  use  of  earned  value  management.  The  revised  policy  (see 
Appendix  A)  established  a  uniform  threshold  for  all  contracts  of  $20  million  and  included 
six  recommendations,  which  significantly  alter  the  EVM  system  used  by  the  DoD  to 
manage  large  contracts. 

This  research  is  specifically  relevant  to  recommendation  number  six,  which  deals 

with  the  requirement  to  perform  a  risk  analysis  on  contracts  to  detennine  if  EVM  would 

benefit  the  project  management.  The  text  of  this  recommendation  is  as  follows: 

The  application  of  EVM  on  cost  or  incentive  efforts,  including  contracts, 
subcontracts,  intra-government  work  agreements,  and  other  agreements 
valued  at  less  than  $20M  is  optional  and  is  a  risk  based  decision  that  is  at 
the  discretion  of  the  program  manager.  A  cost-benefit  analysis  shall  be 
conducted  before  deciding  to  implement  EVM  in  these  situations.  (DoD 
EVM  Policy  Letter,  7  March  2005) 

To  clarify  the  broad  changes,  a  policy  letter  (included  as  Appendix  B),  was  issued  in 
October  2004.  Included  in  the  letter  are  five  recommendations  that  “aim  to  simplify, 
improve,  and  increase  consistency  in  DoD  EVM  policy  and  processes. . .  [and]  . .  .is  a 
reflection  of  modern  day  EVM  best  practices  aimed  at  bringing  EVM  within  DoD  into 
the  21st  Century.”  Among  the  five  recommendations,  the  fifth  recommendation  is  the 
applicable  point  for  this  research  and  it  states:  EVM for  all  firm  fixed  price  contracts, 
subcontracts,  intra-government  work  agreements,  and  other  agreements  will  be  a  risk- 
based  decision  (DoD  EVM  Policy  Letter,  7  March  2005).  However,  a  risk-based 
decision  requires  the  need  to  define  the  risk  and  then  establish  a  decision  process  to 
evaluate  the  identified  risk  factors  associated  with  an  AF  fixed-price  construction  project. 
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Decision  Analysis 

In  order  to  provide  a  decision  maker  with  quantitative  methods  for  exploring  all 
aspects  of  a  decision,  Decision  Analysis  (DA)  is  a  broad  field  of  study  which  provides 
multiple  techniques  to  help  a  decision  maker  consider  all  aspects  and  possible  outcomes 
in  order  to  make  a  more  infonned  decision  (Clemen  and  Reilly,  2001).  DA  does  not 
make  the  decision;  it  structures  the  question,  considerations,  and  options  so  that  the 
decision  maker  has  as  much  available  insight  and  information  as  possible.  When 
considering  DA’s  breadth  of  use,  Arsham  (2006)  states,  “Decision  analysts  provide 
quantitative  support  for  the  decision-makers  in  all  areas  including  engineers,  analysts  in 
planning  offices  and  public  agencies,  project  management  consultants,  manufacturing 
process  planners,  financial  and  economic  analysts,  experts  supporting 
medical/technological  diagnosis,  and  so  on  and  on.” 

DA  Concepts 

Although  DA  often  evokes  thoughts  of  probabilities  and  statistics  along  with 
complex  uncertainty,  the  basic  concept  is  to  structure  the  decision  into  manageable  tasks 
or  sub-decisions.  Figure  2.3  helps  explain  the  process,  showing  steps  that  range  from 
understanding  the  decision  and  its  objectives  through  analysis  and  implementation 
(Clemen  and  Reilly,  2001).  The  uncertainty  that  DA  attempts  to  explore  is  based  on 
empirical  data  in  the  form  of  past  records,  statistics,  observations,  or  experimentation. 
However,  if  no  empirical  data  for  objective  probabilities  exists,  decisions  must  then  be 
based  on  subjective  probabilities  (Haimes,  2004).  Additionally,  DA  relies  on  multiple 
techniques  to  explore  the  elements,  nuances,  and  possible  outcomes  of  a  given  decision. 
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Implement 
the  chosen 
alternative 


Figure  2.3  Decision  Analysis  Process  Flowchart  (Clemen  and  Reilly,  2001) 
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DA  Techniques 

Determining  what  is  important  and  how  the  decision  maker’s  choices  rate  versus 
those  important  criteria  are  the  two  building  blocks  of  decision  analysis  (Kirkwood, 

1997).  DA  is  considered  to  be  “both  a  technique  for  probabilistic  analysis  and  a  tool  for 
exploring  and  evaluating  alternative  courses  of  action”  (Ashley,  Stokes,  and  Perng; 

1988).  The  techniques  available  to  conduct  DA  vary  according  to  the  decision  and  the 
approach  taken  to  evaluate  the  decision.  If  the  focus  is  on  alternatives,  either  Alternative- 
Focused  Thinking  or  Value-Focused  Thinking  would  best  serve  the  decision-maker.  If 
the  decision  relies  on  distinct  relationships  between  factors  and  considerations  must  be 
made  for  the  sequence  or  order  of  the  factors,  influence  diagrams  are  the  most 
appropriate  technique.  However,  if  the  analysis  must  consider  multiple  decisions  and 
includes  chance  events,  a  decision  tree  provides  the  greatest  power  to  evaluate  and 
explore  all  branches  of  the  decision  (Clemen  and  Reilly,  2001).  Each  of  these  techniques 
is  discussed  in  more  detail  in  the  rest  of  this  section. 

Alternative-Focused  Thinking:  Alternative-Focused  Thinking  (AFT)  is  a 
technique  that  allows  the  decision  maker  to  rate  the  alternatives  based  on  the  Decision 
Maker’s  (DM)  assessment  of  the  differences  between  the  alternatives.  AFT  requires  that 
the  decision  be  looked  at  in  terms  of  the  choices  available.  Once  the  alternatives  are 
chosen,  each  one  is  evaluated  based  on  the  same  set  of  criteria  to  detennine  which 
alternative  best  meets  the  decision-maker’s  criteria.  This  technique  is  often  used  when 
the  alternatives  are  established  and  the  criteria  are  well  defined.  A  prime  example  would 
be  the  decision  to  buy  a  new  car.  The  alternatives  are  simply  the  cars  the  DM  is 
interested  in  buying.  The  AFT  methodology  uses  the  DM’s  preferences  to  evaluate  and 
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rank  order  the  cars  based  on  the  value  given  to  price,  color,  engine,  size,  gas  mileage,  etc. 
The  result  is  a  rank  ordering  of  the  cars  considered  and  a  top  choice  that  maximizes  the 
value  structure.  The  problem  arises  when  the  best  alternative  is  not  among  the 
alternatives  evaluated.  Therefore,  AFT  simply  solves  the  problem  by  just  looking  at  the 
apparent  choices  and  picking  the  best  of  what  is  available  (Keeney,  1992). 

Value-Focused  Thinking:  Value-Focused  Thinking  (VFT)  is  very  similar  to  AFT. 
However,  instead  of  evaluating  alternatives  based  on  a  DM’s  distinction  of  the 
alternatives,  the  VFT  process  starts  out  by  asking  the  DM  to  establish  the  criteria  that  are 
most  important  in  the  decision.  For  VFT,  the  process  starts  with  the  best  and  works  to 
make  it  a  reality  with  the  decision  (Keeney,  1992).  Once  the  decision  problem  has  been 
identified,  the  decision-maker  is  asked  to  establish  a  value  hierarchy,  which  forms  the  set 
of  criteria  for  evaluation.  This  process  provides  unique  insight  into  the  decision-maker’s 
values  and  helps  identify  the  important  aspects  of  the  decision.  The  result  is  that  the 
criteria  influence  the  alternatives  rather  than  the  proposed  alternatives  driving  the  criteria 
(Katzer,  2002).  The  resulting  VFT  model  evaluates  the  possible  decision  alternatives 
based  on  the  value  hierarchy  and  returns  a  ranking  of  the  alternatives.  In  the  new  car 
example,  VFT  would  establish  the  value  hierarchy  of  decision  criteria  such  as  price, 
color,  engine,  size,  gas  mileage,  etc.,  before  any  cars  are  evaluated.  Unfortunately,  the 
result  is  simply  an  ordering  of  the  alternatives  and  cannot  be  compared  or  evaluated  to  a 
standard  or  to  each  other.  This  is  the  primary  limitation  of  VFT  and  is  the  reason  it  is  not 
applicable  to  this  research  (Weir,  2005).  This  research  evaluates  each  project 
individually  and  compares  it  to  a  standard,  which  is  not  conducive  to  VFT  methodology. 
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Influence  Diagrams:  Another  decision  analysis  technique  available  to  a  decision¬ 
maker  is  the  influence  diagram.  Influence  diagrams  are  very  useful  because  they 
establish  a  visual  representation  of  the  interaction  between  factors  affecting  a  decision  but 
fall  short  in  the  analysis  of  the  “chronological  structure  and  state  of  uncertain  events” 
(Jeljeli  and  Russell,  1995).  Used  often  to  evaluate  investment  alternatives,  influence 
diagrams  create  an  effective  illustration  for  computer  analysis;  they  also  depict 
relationships  between  decisions  and  risk,  as  well  as  provide  a  clear  characterization  of 
probabilistic  interdependencies  (Ashley  et  ah,  1988).  However,  influence  diagrams  have 
many  pitfalls,  such  as  confusing  them  with  flow  diagrams  and  a  tendency  to  build  a 
circular  path  in  the  diagram  (Clemen  and  Reilly,  2001).  These  pitfalls  and  the  inability  to 
adequately  explore  all  the  risk  scenarios  of  an  USAF  construction  project  make  the 
influence  diagram  inappropriate  for  this  research. 

Decision  Trees:  Decision  trees  are  similar  to  influence  diagrams  but  provide  a 
much  greater  level  of  detail  into  the  possible  outcomes  (Clemen  and  Reilly,  2001).  They 
provide  a  breakdown  of  the  component  elements  of  a  decision,  integrate  uncertainty,  and 
yield  an  outcome;  this  allows  for  the  optimization  of  the  expected  value,  thereby 
exploring  all  possible  risk  scenarios  associated  with  the  uncertainty  (Jeljeli  and  Russell, 
1995).  The  power  of  a  decision  tree  is  to  integrate  multiple  decisions  and  multiple 
chance  events  into  a  single  structure;  when  solved,  it  yields  a  result  that  considers  the 
probability  and  outcome  of  the  options  of  a  chance  event  and  considers  the  benefit  of 
each  choice  in  the  decision.  The  other  benefit  of  a  decision  tree  is  the  ability  to  overcome 
the  limitation  of  sequence  or  order  of  chance  events  (Clemen  and  Reilly,  2001).  Unlike 
influence  diagrams,  the  order  of  events  does  not  affect  the  outcome  of  the  result  (Clemen 


28 


and  Reilly,  2001).  Decision  trees  are  used  for  a  variety  of  decisions,  such  as  whether  to 
invest  in  a  risky  stock  venture  to  the  likelihood  of  a  hurricane  hitting  a  certain  area  and 
the  decision  whether  to  evacuate  or  stay.  The  limiting  factor  of  a  decision  tree  is  that 
with  multiple  chance  nodes  and  multiple  outcomes  the  exploration  of  every  possible 
outcome  grows  the  tree  exponentially  (Weir,  2005).  For  instance,  a  tree  with  4  sequential 
chance  events  and  3  possible  outcomes  per  event  results  in  8 1  (3  raised  to  the  4th  power) 
branches.  By  adding  2  more  chance  events,  the  number  of  branches  grow  to  729.  Due  to 
the  nature  of  construction  risk  factors  and  variability,  a  decision  tree’s  ability  to  evaluate 
multiple  levels  of  risk  for  each  chance  event  lends  itself  best  to  this  research. 

Decision  Analysis  Summarized:  Regardless  of  the  technique,  DA  allows  for  the 
exploration  of  various  facets  of  a  decision.  Whether  the  technique  is  considering  the 
DM’s  values  or  investigating  the  possible  outcomes  of  chance  events,  DA  provides  the 
DM  with  additional  insight  into  the  decision  and  the  choices  to  be  made.  In  this  research, 
the  chance  events  are  based  on  risk  and  the  analysis  of  those  factors,  which  can  affect  the 
outcome  (i.e.,  the  decision).  The  power  of  DA  is  the  development  of  a  cumulative  risk 
profde  of  the  factors,  which  shows  the  possible  outcomes  and  expected  value,  thereby 
providing  insight  into  the  decision  (Kirkwood,  1997).  As  such,  the  next  step  is  to 
understand  the  risk  related  to  construction  projects. 

Risk  Factors 

Risk  is  simplistically  described  as  uncertainty;  if  everything  were  known  about  a 
project,  there  would  be  no  risk.  To  be  more  precise  though,  risk  has  two  distinct 
elements:  the  potential  loss  due  to  the  risk  coupled  with  the  likelihood  of  the  risk 
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(Kendrick,  2003).  EVM  can  identify  cost  and  schedule  problems  in  a  project,  so  what 
remains  is  the  question  of  what  makes  a  construction  project  risky.  This  is  the  key  issue 
that  this  research  will  attempt  to  identify  and  quantity.  Therefore,  a  model  is  necessary 
that  will  evaluate  construction  projects  to  detennine  the  level  of  risk  or  uncertainty 
associated  with  it  and  determine  whether  EVM  is  an  appropriate  tool  to  use  to  monitor 
and  control  those  uncertainties. 

Fundamentals  of  Risk 

Merriam- Webster  (2006)  defines  risk  as,  “the  possibility  of  loss  or  injury, 
someone  or  something  that  creates  or  suggests  a  hazard,  or  the  chance  of  loss  or  the  perils 
to  the  subject  matter  of  an  insurance  contract;  also :  the  degree  of  probability  of  such 
loss.”  Relating  this  to  construction  risk,  Smith  and  Merritt  (2002)  state  that  risk  is  “the 
possibility  that  an  undesirable  outcome-or  the  absence  of  a  desired  outcome-disrupts  your 
project.”  Decision  analysis  may  relate  risk  more  to  the  analytical  definition  of  risk  as  “a 
measure  of  the  probability  and  severity  of  adverse  effects”  (Haimes,  2004).  Taking  these 
definitions  into  consideration,  this  research  must  identify  those  factors,  which  create  risk, 
determine  the  cost  impact  of  those  factor’s  effects,  and  quantity  them  in  relation  to 
construction  projects. 

Managing  Risk 

Risk  management  can  be  defined  as  “the  art  and  science  of  identifying,  assessing 
and  responding  to  project  risk”  (Wideman,  1992).  Although  there  are  many  approaches 
and  techniques  to  risk  management,  a  top-level  look  would  divide  the  field  into  micro 
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and  macro.  Micro  risk  management  deals  with  risk  on  an  individual  basis,  while  macro 
risk  management  looks  at  statistical  averages  and  aggregate  samples.  The  power  is  in 
using  the  macro  perspective  to  gain  insight  and  knowledge  about  the  whole  while  using 
the  micro  view  to  asses  the  individual  risks  for  each  project  (Kendrick,  2003). 

Existing  Research 

As  discussed  in  the  following  paragraphs,  previous  studies  have  explored  the 
uncertainty  associated  with  construction.  The  financial  risk  to  both  the  owner  and  the 
contractor  require  that  much  care  be  taken  before  entering  into  such  a  venture. 
Unfortunately,  this  care  often  involves  only  a  brief  analysis  based  on  the  PM’s  experience 
of  previous  projects  instead  of  a  detailed  evaluation  and  quantification  of  the  proposed 
project’s  risk  and  uncertainty  (Mustafa  and  Al-Bahar,  1991).  Three  relevant  studies  in 
the  area  of  construction  project  related  to  risk  identification  and  assessment  will  be 
discussed.  These  studies  cover  risk  related  to  environmental  remediation  liability,  the 
classification  of  a  risk  factor  hierarchy  related  to  construction  in  foreign  countries,  and  a 
unique  perspective  related  to  the  quantification  of  risk  by  using  common  language. 

Environmental  Remediation 

The  most  relevant  of  these  studies  is  by  Jeljeli  and  Russell  (1995),  who  quantified 
risk  factors  associated  with  environmental  remediation  projects.  Using  influence 
diagrams  and  decision  trees,  they  used  risk  assessment  to  quantify  the  liability  faced  by  a 
contractor  bidding  on  a  project.  By  surveying  80  companies  involved  in  environmental 
remediation,  they  identified  and  rank  ordered  53  risk  factors  based  on  respondents’ 


31 


perceptions  of  importance.  Table  2. 1  shows  the  rankings  from  the  survey.  They 
concluded  that  41  of  these  factors  affect  the  likelihood  of  a  liability  claim.  Five  subject 
matter  experts,  who  also  provided  the  probability  of  each  factor,  validated  the  factors  and 
their  related  influence.  Figure  2.4  shows  the  interaction  of  these  factors.  Thus,  Jeljeli 
and  Russell  (1995)  produced  a  quantitative  model  capable  of  integrating  “knowledge  and 
experience  acquired  from  multiple  experts,  project-specific  knowledge,  problem-solving 
and  decision  techniques”  to  estimate  liability. 
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Table  2.1  Rank  Order  of  Variables  (Jeljeli  and  Russell,  1995) 


Risk  Factor 

Rank 

Risk  Factor 

Rank 

Contract  clauses 

1 

Reliability  of  technology 

28 

Private  Owner  indemnification 

2 

Constructability  of  design 

29 

Government  indemnification 

3 

Demonstrated  performance  of  technology 

30 

Type  of  project 

4 

Contractor  past  performance 

31 

Type  of  work 

5 

Source  of  contamination 

32 

Contract  type 

6 

Pathways  of  contamination 

33 

Nature  of  contaminants 

7 

Owner  management  expertise 

34 

Potential  third-party  plaintiffs 

8 

Contractor  reputation 

35 

Threat  to  nearby  community 

9 

Partnering  agreement 

36 

Contractor  experience 

10 

Designer  financial  stability 

37 

Designer  experience 

11 

Implementation  of  technology 

38 

Scope  of  engineering  services 

12 

Owner  reputation 

39 

Performance  of  technology 

13 

Designer  reputation 

40 

Accuracy  of  design  documents 

14 

Surety  bonds 

41 

Insurance  coverage 

15 

Designer  past  performance 

42 

Scope  of  construction 

16 

Time  required  to  reach  objectives 

43 

Effectiveness  of  technology 

17 

Useful  life  of  technology 

44 

Potential  contract  party  plaintiffs 

18 

Area  of  project  site 

45 

Quality  of  design 

19 

Ease  of  start-up 

46 

Extent  of  contamination 

20 

Ease  of  installation 

47 

Designer  management  expertise 

21 

Time  for  implementation  of  technology 

48 

Owner  financial  stability 

22 

Cost  of  design  effort 

49 

Contractor  management  expertise 

23 

Economy  of  design 

50 

Usability  of  design  documents 

24 

Equipment  requirements 

51 

Administrative  and  criminal  penalties 

25 

Labor  requirements 

52 

Threat  to  workers  during  construction 

26 

Material  requirements 

53 

Contractor  financial  stability 

27 
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Analytic  Hierarchy 

Mustafa  and  Al-Bahar  (1991)  assessed  unanticipated  risks  related  to  construction 
projects  and  found  a  hierarchy  of  risk  categories  and  factors  similar  to  Jeljeli  and  Russell 
(1995).  Mustafa  and  Al-Bahar  (1991)  contend  that  current  models  fall  into  one  of  two 
classifications:  classical  and  conceptual.  The  classical  is  categorized  by  probability 
theory  and  analysis  performed  through  influence  diagrams  and  Monte  Carlo  simulations. 
On  the  other  hand,  the  conceptual  relies  on  subjective  analysis  such  as  the  “fuzzy  set 
analysis”  perfonned  by  Kangari  and  Riggs  (1989).  Mustafa  and  Bahar  (1991)  utilized  an 
Analytic  Hierarchy  Process  (AHP)  that  involves  “a  multi-criteria  decision  analysis 
methodology  that  allows  subjective  as  well  as  objective  factors  to  be  considered.”  Figure 
2.5  shows  the  hierarchical  format  of  the  risk  factors  developed  by  Mustafa  and  Al-Bahar 
(1991).  Although  there  is  no  mention  of  how  the  risk  factors  were  derived  or  developed, 
the  categories  established  fall  in  line  with  previous  research. 
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Structure 

-  Landslide 

-  Damage  to 

-  Fire 

Equipment 

-  Wind 

-  Labor  Injuries 

Damage 

-  Material  and 

-  Lighting 

Equipment 

-  Weather 
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-  Inflation 

-  Changes  in 
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Regulations 

Client 

-  War  and  Civil 
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Disorder 
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-  Financial 
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Default  of 

and  Their 
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Approval 

-  Non- 

-  Pollution  and 
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Safety  Rules 
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-  Embargoes 
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-  Labor  Dispute 
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-  Labor 

Design 
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-  Different  Site 

Omissions 

Conditions 

-  Inadequate 

-  Defective 

Specifications 

Work 

-  Design 

-  Equipment 

Changes 

Failure 

Figure  2.5  Hierarchy  of  Risk  Factors  (Mustafa  and  Al-Bahar,  1991) 


Linguistics 

Kangari  and  Riggs  (1989)  present  an  innovative  approach  to  quantifying  risk. 
Their  approach  is  to  utilize  linguistic  terms  or  words  and  to  systematically  assign  a 
quantitative  scale  to  represent  those  terms.  For  instance,  an  expert  may  classify  risk  as 
low,  medium,  or  high.  Kangari  and  Riggs  (1989)  extend  that  expert  definition  to  a  scale 
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of  1  to  10  in  which  the  expert  knows  with  100%  certainty  that  1  is  low,  5  is  medium,  and 
10  is  high.  The  uncertainty  in  the  level  of  risk  is  represented  by  confidence  intervals  that 
include  2  through  4  and  6  through  9  as  shown  in  Figure  2.6.  These  intervals  represent  an 
expert’s  confidence  in  classifying  a  variable  as  low,  medium,  or  high.  For  example,  a 
scale  value  of  4  can  represent  either  a  low  risk  with  10%  confidence  or  a  medium  risk 
with  70%  confidence.  In  this  manner,  Kangari  and  Riggs  (1989)  converted  the  subjective 
assessment  of  risk  into  an  objective  mathematical  model. 


Figure  2.6  Confidence  Interval  Development  (Kangari  and  Riggs,  1989) 
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Conclusion 


This  chapter  reviewed  the  applicable  literature  in  the  areas  of  construction 
management,  earned  value  management,  decision  analysis,  and  risk.  The  content  of  this 
literature  review  will  be  used  to  form  the  foundation  for  the  following  chapter,  which  will 
explain  the  methodology  used  to  construct  the  decision  tree  model.  The  model  will 
attempt  to  quantify  various  aspects  of  construction  management  using  decision  analysis 
techniques  to  assess  risk  in  an  effort  to  determine  if  earned  value  management  is 
applicable  to  construction  projects. 
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CHAPTER  3 


METHODOLOGY 


The  goal  of  this  research  was  to  develop  a  decision  tool  to  enable  decision-makers 
to  make  a  “risk-based  decision”  to  determine  the  applicability  of  an  Earned  Value 
Management  System  (EVMS)  for  fixed-price  construction  contracts  within  the  Air  Force. 
To  meet  this  objective,  the  quantification  of  risk  must  be  addressed  and  then  the 
development  of  the  possible  risk  scenarios  must  be  explored.  To  address  these  risk 
issues,  the  Delphi  method  was  initially  used  to  gather  and  consolidate  expert  opinions. 
Decision  Analysis  (DA),  specifically  the  application  of  a  decision  tree  methodology,  was 
used  to  evaluate  the  multiple  variables  of  risk  inherent  in  construction.  Therefore,  this 
chapter  details  the  Delphi  method  and  decision  tree  model  used  in  this  research. 

Delphi  Method 

The  Delphi  method  is  a  process  by  which  objective  data  that  is  unavailable  in 
quantitative  form  can  be  acquired  through  the  use  of  multiple  expert  opinions  to  form  a 
consensus  (Clayton,  1997).  ft  is  an  iterative  process  to  acquire  information  from  experts 
about  a  particular  problem.  To  gain  a  better  understanding  of  its  applicability  to  this 
research,  this  section  provides  some  fundamental  background  information  related  to  the 
Delphi  method. 
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Historical  Background 

The  name  Delphi  comes  from  mythology  and  references  the  predictive  powers  of 
the  Greek  necromancers  at  the  oracle  at  Delphi  (Gordon,  1994).  The  method  developed 
from  a  project  conducted  in  the  1950s  by  the  RAND  Corporation,  which  was  hired  to 
develop  a  list  of  targets  in  the  United  States  that  the  Soviet  Union  would  strike  in  the 
event  of  a  nuclear  attack  (Linstone  and  Turoff,  1975).  Gordon  credits  the  beginnings  of 
Delphi  to  the  RAND  efforts:  “Olaf  Helmer,  Nicholas  Rescher,  Nonnan  Dalkey,  and 
others  at  RAND  developed  the  Delphi  method,  which  was  designed  to  remove 
conference  room  impediments  to  a  true  expert  consensus.”  Rather  than  rely  on  subjective 
data  or  a  single  expert,  the  concept  was  to  gain  consensus  among  experts  as  to  which  sites 
were  most  likely  to  be  targeted.  Known  as  “Project  Delphi,”  the  RAND  study  became 
the  basis  for  the  Delphi  method  (DeReus,  2004). 

Basic  Process 

The  process  encompasses  a  systematic  approach  to  accomplish  the  objective  of 
gaining  expert  opinion  to  acquire  quantitative  data.  Building  on  the  original  three  steps, 
other  research  has  modified  the  process.  Several  methods  will  be  discussed;  the  strengths 
and  weaknesses  identified,  and  the  steps  will  be  explained. 

The  original  process  consisted  of  a  three  step  approach.  Gunaydin  (1999) 
identifies  the  basic  elements  of  the  original  process  as  “(1)  structuring  of  information 
flow,  (2)  feedback  to  the  participants,  and  (3)  anonymity  for  the  participants.”  The 
process  provides  a  better  flow  of  communications  while  eliminating  the  pitfalls  of  group 
dynamics  to  maintain  focus  on  the  issue  (Jensen,  1981). 
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An  evolution  of  the  three-step  process  is  the  five-step  approach  presented  by 
DeReus  (2004)  and  shown  in  Figure  3.1  Summarized  from  information  provided  by 
Ryerson  University,  DeReus  (2004)  offers  the  following  explanations  of  process  steps. 
Step  one  is  simply  the  identification  of  the  problem,  which  reinforces  the  need  to 
understand  the  problem  before  proceeding.  Step  two  requires  the  identification  and 
selection  of  experts.  The  intent  is  to  elicit  input  from  individuals  knowledgeable  and 
experienced  in  the  field  related  to  the  problem.  This  continues  the  focus  on  the  problem 
identified  in  step  one.  Having  identified  the  Subject  Matter  Experts  (SMEs),  Steps  three 
and  four  are  a  repeating  loop.  Steps  three  and  four  entail  the  fundamentals  of  steps  two 
and  three  identified  by  Gunaydin  (1999)  in  the  original  process.  By  performing  repetitive 
iterations,  the  process  ensures  participants  gain  feedback  for  each  loop.  Steps  three  and 
four  are  iterative  processes  that  repeat  in  order  to  reach  consensus  by  the  SMEs 
concerning  the  identified  problem.  Step  five  translates  the  data  gained  through  the 
Delphi  Method  (DeReus,  2004). 
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Figure  3.1  Basic  Process  (DeReus,  2004) 
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As  an  expansion  on  the  original  three-step  process  and  to  further  refine  the 
process,  Fowles  (1978)  offers  the  following  process  steps: 

1 .  Formation  of  a  team  to  undertake  and  monitor  a  Delphi  on  a  given  subject. 

2.  Selection  of  one  or  more  panels  to  participate  in  the  exercise.  Customarily, 
the  panelists  are  experts  in  the  area  to  be  investigated. 

3.  Development  of  the  first  round  Delphi  questionnaire 

4.  Testing  the  questionnaire  for  proper  wording  (e.g.,  ambiguities,  vagueness) 

5.  Transmission  of  the  first  questionnaires  to  the  panelists 

6.  Analysis  of  the  first  round  responses 

7.  Preparation  of  the  second  round  questionnaires  (and  possible  testing) 

8.  Transmission  of  the  second  round  questionnaires  to  the  panelists 

9.  Analysis  of  the  second  round  responses  (Steps  7  to  9  are  reiterated  as  long  as 
desired  or  necessary  to  achieve  stability  in  the  results.) 

10.  Preparation  of  a  report  by  the  analysis  team  to  present  the  conclusions  of  the 
exercise 

Although  similar  to  the  basic  process,  Fowles’  (1978)  process  provides  additional  detail 
and  refinement  to  improve  the  effectiveness  of  the  method.  The  key  element  added  by 
Fowles  (1978)  is  that  of  testing.  He  proposes  testing  of  the  first  round  questionnaire  and 
possible  testing  of  questionnaires  in  subsequent  rounds  before  proceeding. 
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General  Areas  of  Application 

The  areas  in  which  the  Delphi  method  has  been  used  vary  greatly.  From  its  initial 
use  as  a  prediction  tool,  it  has  developed  into  a  tool  that  has  broad  applicability  across 
many  disciplines.  Linstone  and  Turoff  (1975)  contend  that  it  is  used  “in  the  field  of 
planning,  Normative  Forecasting,  Planning  Programming  Budgeting  Systems  (PPBS), 
and  Cost-Effectiveness  or  Cost-Benefit  Analyses.”  In  his  examination  of  Delphi,  DeReus 
(2004)  presents  its  extensive  use  in  engineering  and  the  physical  sciences  as  well  as 
economics  and  business.  The  benefit  of  Delphi  is  its  effectiveness  when  data  is  too  costly 
or  time  inhibitive  to  collect  (DeReus,  2004) 

Strengths 

The  basic  premise  of  Delphi  is  consistent  regardless  of  the  exact  approach  that  is 
used:  expert  opinion  and  consensus  is  more  accurate  and  more  reliable  than  that  of  a 
single  individual.  The  ability  to  construct  dependable  data  from  subjective  responses  to 
reach  quantifiable  results  demonstrates  the  power  of  the  Delphi  method  (Gunaydin, 

1999).  The  method’s  primary  strength  is  its  ability  to  draw  on  and  form  a  consensus  of 
experts  without  requiring  those  experts  to  come  together  and  meet.  The  lack  of  excessive 
cost  and  planning  typically  associated  with  bringing  individuals  together  are  other 
strengths  of  the  Delphi  method  (DeReus,  2004).  Mattingley-Scott  (2006)  offers  these 
benefits  regarding  the  method:  “rapid  consensus,  participants  can  reside  anywhere  in  the 
world,  coverage  of  [a]  wide  range  of  expertise,  [it]  avoids  groupthink,  [and  its  ability  of] 
forecasting  a  specific,  single-dimension  question.”  A  further  benefit  is  the  iterative 
nature  of  the  process  which  ultimately  forms  the  benefit  of  consensus  (DeReus,  2004). 
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In  addition,  the  process  is  flexible  enough  to  enable  variations  in  the  investigative 
portion.  Gordon  (1994)  contends  that  although  the  initial  and  final  steps  may  be  similar 
in  different  processes,  the  questionnaire  phase  can  vary  greatly.  In  fact,  Gordon  (1994) 
offers  three  distinct  methods  that  can  be  used  to  question  the  SMEs.  One  method  is  to 
simply  use  the  first  round  data  in  quantitative  decision  models.  The  subsequent  analysis 
of  the  model  validates  the  expert  opinion  and  determines  if  the  differences  in  the  expert 
opinions  are  statistically  significant.  The  second  method  is  one-on-one  interviews  in 
which  the  interactive  nature  allows  for  more  flexibility  than  with  a  questionnaire.  In 
addition,  interviews  can  be  directed  based  on  interviews  with  previous  experts,  which 
Gordon  (1994)  calls  “Feed-Forward.”  Although  interviews  are  therefore  different,  the 
objective  is  reached.  Finally,  Gordon  (1994)  offers  group  meetings  as  the  third  method. 
This  conflicts  with  the  original  premise  of  anonymity,  but  it  is  acceptable  in  situations 
were  the  topic  of  discussion  is  not  sensitive  or  controversial  and  will  not  elicit  strong 
responses  which  may  drive  the  focus  of  the  group  (Gordon,  1994). 

The  Delphi  method  offers  many  benefits  to  the  researcher,  some  of  which  were 
mentioned  above.  Its  ability  to  leverage  multiple  expert  opinions  while  eliminating  the 
expense  to  bring  those  individuals  together,  its  iterations  to  form  consensus,  and  its 
flexibility  make  it  an  excellent  tool.  However,  no  method  is  without  criticism  and  those 
ideas  will  be  expressed  in  the  next  section. 

Weaknesses 

The  criticisms  of  Delphi  fall  in  to  three  main  categories:  selection  of  the  experts, 
bias  imposed  by  the  research,  and  reliability.  Clayton  (1997)  contends  that  expertise  is 
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not  a  measurable  quantity;  therefore,  the  selection  of  experts  is  a  very  subjective  process. 
Furthermore,  an  individual’s  knowledge  and  experience  related  to  a  particular  subject 
does  not  necessarily  correlate  to  the  quality  of  the  input  provided;  additionally, 
experience  may  prevent  alternatives  outside  of  the  nonnal  paradigms  related  to  that 
subject  (Mattingley-Scott,  2006). 

The  second  criticism  relates  to  the  researcher.  DeReus  (2004)  points  out  that  the 
opportunity  for  bias  exists  through  the  researcher’s  examination,  summation,  and 
formulation  of  each  round  of  responses  and  questions.  In  addition,  the  simplistic  nature 
of  the  Delphi  method  can  create  a  false  sense  of  ease  and  diminish  the  critical  nature 
needed  in  the  refinement  of  each  round  of  questions  and  responses  (Clayton,  1997). 

The  third  criticism  is  the  reliability  of  the  process  itself.  Although  this  relates 
back  to  the  selection  of  the  experts,  the  question  is  whether  the  data  is  reproducible  with 
similar  experts  responding  to  the  same  questionnaires  (DeReus,  2004).  In  addition,  the 
issue  of  reliability  permeates  into  the  depth  of  investigation  concerning  differences  in 
responses  between  rounds  (Mattingley-Scott,  2006).  The  result  of  the  criticism  is  that 
with  any  method,  care  must  be  taken  to  ensure  the  scientific  method  is  followed  and  to 
understand  that  all  results  are  subject  to  interpretation  and  discussion. 

Decision  Tree 

A  decision  tree  is  designed  to  show  decisions,  risk,  cost,  and  probabilities  in  a 
graphical  representation.  As  the  name  implies,  the  tree  represents  the  flow  of  the 
relationship  between  choices  available  to  a  decision  maker  and  the  possible  outcomes  of 
chance  events  which  are  out  of  the  decision  maker’s  control  (Haimes,  2004).  This 
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research  relied  on  two  methods  to  construct  the  decision  tree.  The  first  method  used  DPL 


4.0  (decision  analysis  software)  and  failed  due  to  excessive  computational  time 
requirements;  therefore,  Microsoft  Excel  Visual  Basic  was  used.  After  a  general 
discussion  of  the  strengths  and  weaknesses  associated  with  the  use  of  decision  trees,  both 
methods  will  be  explained  in  more  detail. 

General  Strengths  and  Weaknesses 

Decision  trees  have  several  strengths  and  weaknesses.  The  primary  strengths  are 
that  it  can  handle  multiple  types  of  input,  can  be  validated  with  statistical  methods,  and 
can  handle  large  amounts  of  data  quickly  (Weir,  2005).  The  weaknesses  range  from  the 
complexity,  which  can  quickly  escalate,  to  the  graphical  nature  of  the  diagram. 
Complexity  can  quickly  become  a  problem  because  of  the  exponential  growth  of  the  tree 
when  additional  chance  events  are  added.  This  also  leads  to  a  cumbersome  graphical 
representation  that  is  neither  easily  viewed  nor  comprehended.  The  individual  strengths 
and  weaknesses  of  DPL  and  Microsoft  Excel  Visual  Basic  will  also  be  discussed  in  the 
following  sections. 

DPL 

DPL  is  a  computer  program  developed  by  Applied  Decision  Analysis  LLC  for 
modeling  influence  diagrams  and  decision  trees  (Borison  et  ah,  1998).  In  addition,  the 
program  offers  the  advantage  of  being  able  to  convert  influence  diagrams  into  decision 
trees.  This  gives  the  modeler  the  capability  to  graphical  represent  a  decision  through  the 
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use  of  an  influence  diagram,  which  is  relatively  simple.  The  program  will  automatically 
convert  the  relationships  described  in  the  influence  diagram  into  a  decision  tree. 

DPL  also  provides  the  capability  to  conduct  sensitivity  analysis.  Several 
processes  are  available  to  help  the  decision  maker  understand  the  range  of  values  and 
their  impact.  The  first  is  a  tornado  diagram,  which  indicates  the  range  over  which  the 
variables  can  change  and  the  impact  of  those  changes  on  a  particular  decision.  A  variable 
with  a  greater  the  range  is  shown  by  a  longer  bar  on  the  tornado  diagram.  The  color  of 
the  bar  for  a  given  variable  may  change  as  well;  this  demonstrates  where  the  choice  of 
alternatives  would  change  if  the  variable  changed  to  that  extent.  For  example,  the 
decision  to  produce  a  product  to  sell  will  depend  on  factors  such  as  market  size,  the 
companies’  share  of  that  market,  selling  price,  fixed  costs,  and  variable  costs.  An 
analysis  of  these  risk  factors  will  reveal  that  share  of  the  market  fluctuates  to  a  greater 
extent  than  variable  price.  Therefore,  market  share  should  be  examined  and  tracked  more 
than  the  variable  price  to  manage  the  risk  from  producing  the  specified  product.  The 
tornado  diagram  of  this  example  is  shown  in  Figure  3.2. 
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Tornado  Diagram 
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Figure  3.2  Tornado  Diagram  (Borison  et  al.,  1998) 


Another  feature  of  DPL  is  the  risk  profile.  As  Borison  et  al.  (1998)  put  it,  “risk 
profiles  graphs  the  range  of  outcomes  under  alternative  strategies  [and]  help  understand 
which  strategies  offer  the  most  risk  and  which  offer  the  most  potential.”  As  in  the 
previous  product  example,  decisions  affect  the  risk  factors  and  the  probability  of  an 
outcome  from  that  decision  is  demonstrated  by  the  risk  profile.  An  example  of  a  risk 
profile  is  shown  in  Figure  3.3. 
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Figure  3.3  Risk  Profile  (Borison  et  al.,  1998) 


Unfortunately,  DPL  is  limited  by  the  number  of  chance  events  it  can  evaluate.  In 
a  test  of  the  program  using  13  risk  variables  with  a  range  of  three  possible  scenarios,  the 
computer  program  failed  to  produce  an  output.  Subsequent  runs  with  fewer  variables 
produced  the  same  result.  For  this  reason,  DPL  could  not  be  used  for  this  research. 
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Microsoft  Excel 


Using  Microsoft  Excel  and  Visual  Basic  Editor,  a  computer  program  can  be 
written  which  will  evaluate  and  calculate  the  risk  scenarios  demonstrated  in  a  decision 
tree  model.  Although  Microsoft  Excel  allows  for  the  computation  of  the  nearly  two 
million  scenarios  likely  in  this  research,  the  program  has  its  limitations.  The  program 
does  not  have  a  preset  sensitivity  analysis  capability  similar  to  that  available  through 
DPL  4.0  software.  The  tornado  diagram  that  provides  insight  into  the  range  of  the  risk 
factors  is  not  available;  however,  subroutines  can  be  written  to  track  the  information 
needed  to  graph  the  probability  distribution  and  cumulative  probability.  Examples  of 
each  are  shown  in  Figures  3.4  and  3.5.  Although  the  lack  of  a  sensitivity  analysis  feature 
is  a  limiting  factor,  the  capability  of  Microsoft  Excel  to  compute  the  full  range  of  risk 
scenarios  makes  it  superior  to  DPL  4.0.  Thus,  the  Microsoft  Excel  decision  tree  model 
provides  the  basis  for  this  research. 
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Probability  Probability 


S 

Figure  3.4  Probability  Distribution  ( NIST/SEMATECH ,  2006) 
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Figure  3.4  Cumulative  Probability  {NIST/SEMATECH ,  2006) 


52 


Conclusion 


The  methodology  used  in  this  research  attempts  to  quantify  the  risk  associated 
with  construction.  The  risk  factors  were  gleaned  from  relevant  literature,  confirmed  and 
expanded  for  likelihood  and  impact  through  the  Delphi  method  with  Subject  Matter 
Experts.  Using  the  data  acquired,  a  decision  tree  model  was  developed  using  DPL  and 
subsequently  in  Microsoft  Excel.  The  purpose  is  to  develop  a  model,  which  will  assist  a 
decision  maker  in  determining  the  applicability  of  EVM  for  construction.  The  success  of 
this  effort  and  the  further  exploration  and  analysis  of  the  model  will  be  discussed  in  the 
next  chapter. 
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CHAPTER  4 


RESULTS  AND  DISCUSSION 


This  chapter  presents  the  results  of  applying  the  methodology  explained  in 
Chapter  3 .  After  first  describing  how  the  appropriate  data  was  collected,  the  chapter 
discusses  the  formulation  of  the  model  developed  in  this  research.  It  also  explains  how 
the  model  was  adapted  in  response  to  inputs  from  Subject  Matter  Expert  (SME)  opinion. 
The  overall  result  is  the  development  of  a  decision  tool  that  will  help  Air  Force  (AF) 
personnel  manage  construction  projects.  The  tool  helps  project  managers  (PMs)  identify 
the  risks  associated  with  a  project  and  determine  whether  that  risk  warrants  the  use  of 
earned  value  management  (EVM).  The  model  is  based  on  a  decision  tree  which  uses 
either  the  aggregate  assessment  of  opinions  from  SMEs  or  the  PM’s  specific  knowledge 
of  the  project  to  predict  expected  cost  growth.  The  decision-maker  then  has  the  choice  of 
using  EVM  during  the  project  to  anticipate  problems  and  help  mitigate  risk. 

Data  Collection  Efforts 

The  data  for  this  research  effort  was  collected  through  the  use  of  the  Delphi 
method.  As  discussed  in  Chapter  3,  the  Delphi  method  utilizes  an  iterative  process.  The 
results  from  applying  each  of  these  steps  are  expressed  in  the  following  sections.  The 
Delphi  method  was  appropriate  for  this  research;  it  attempted  to  gamer  expert  opinion 
from  multiple  sources  as  a  way  to  detennine  an  aggregate  assessment  of  the  current  risks 
experienced  by  PMs  in  the  construction  field. 


54 


Step  One 

The  first  step  in  the  Delphi  method  is  to  assess  the  problem.  The  problem  arises 
from  the  revised  policy  that  requires  a  “risk-based  decision”  to  detennine  whether  EVM 
should  be  used  for  projects  valued  at  less  than  $20  million.  Since  most  military 
construction  (MILCON)  projects  fall  into  this  category,  this  revised  policy  is  applicable 
PMs  in  the  civil  engineering  organization.  Therefore,  the  problem  to  be  assessed  in  this 
step  is  to  identify  the  risks  associated  with  AF  construction  projects. 

The  problem  arises  from  a  need  to  perform  a  risk-based  decision  to  determine 
whether  sufficient  cost  or  schedule  risk  exists  to  necessitate  EVM  for  firm-fixed  price 
construction  contracts.  Cost  and  schedule  risk  create  the  cost  growth  associated  with  AF 
construction  projects.  The  relevant  cost  growth  data  is  provided  in  the  Automated  Civil 
Engineering  System  (ACES).  The  information,  however,  is  incomplete  and  difficult  to 
extract.  Therefore,  the  Delphi  method  was  chosen  to  overcome  this  lack  of  quantifiable 
cost  growth  data  and  focus  on  the  quantification  of  the  associated  risk  factors,  which  are 
inherently  subjective  rather  than  objective. 

Step  Two 

Step  two  of  the  Delphi  method  is  to  identify  appropriate  experts.  For  this 
research,  this  meant  finding  experts  associated  with  management  of  construction  projects 
in  the  AF.  The  choice  was  to  consult  members  of  the  Air  Force  Center  for 
Environmental  Excellence  (AFCEE)  at  Brooks  City  Base  in  San  Antonio,  Texas.  Five 
AFCEE  project  managers  were  selected  based  on  their  first  hand  knowledge  of 
construction  and  their  contract  management  experience  in  project  modifications  and 
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change  orders.  These  individuals  represent  years  of  construction  management  experience 
from  which  to  draw  professional  expertise.  Their  recent  project  management  experience 
includes  $1.2  billion  in  construction  in  2002,  over  $1.2  billion  in  Iraq  reconstruction  in 
2004,  projections  of  $2  to  $2.5  billion  in  2005  and  2006  for  Iraq  reconstruction,  and  most 
recently  the  rebuilding  effort  at  Keesler  Air  Force  Base,  Mississippi,  following  the 
devastation  due  to  Hurricane  Katrina  (AFCEE  Website,  2006).  To  ensure  the  anonymity 
of  these  SMEs,  they  will  be  referred  to  only  by  a  letter  designation. 

Step  Three 

Step  three  is  the  repetitive  loop  of  querying  the  SME  and  consolidating  the  results 
to  fonn  a  consensus;  it  was  conducted  in  three  phases.  The  first  phase  reviewed  the 
relevant  literature  to  develop  applicable  construction  risk  factors.  The  second  phase 
asked  the  SMEs  to  validate  the  relevant  risk  factors  related  to  AF  construction  projects, 
while  the  third  phase  detennined  the  likelihood  and  impact  of  the  identified  risk  factors. 
The  three  phases  were  conducted  in  an  iterative  process  and  are  described  in  the 
following  paragraphs. 

Phase  One:  The  literature  review  included  previous  research  that  had  identified 
risk  factors  for  specific  types  of  construction.  For  example,  Mustafa  and  Al-Bahar 
(1991)  used  the  Analytic  Hierarchy  Process  (AHP)  to  quantify  the  risks  associated  with 
construction  projects  in  foreign  countries.  They  established  the  six  categories  and  32 
factors  previously  shown  in  Figure  2.5.  Asa  reminder,  the  categories  were  Acts  of  God, 
Physical,  Financial  and  Economic,  Political  and  Environmental,  Design,  and  Job  Site- 
Related.  Since  the  current  research  effort  is  focused  on  projects  in  the  United  States  in 
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the  public  sector,  the  Financial  and  Economic  along  with  the  Political  and  Environmental 
factors  were  not  applicable.  Of  the  remaining  24  risk  factors,  the  ones  used  in  this 
research  were  weather,  differing  site  conditions,  design  accuracy,  and  design  changes. 

By  expanding  their  definitions,  these  four  accurately  encompassed  the  remaining  20  and 
were,  therefore,  not  listed  independently.  The  four  were  applicable  and  relevant  to  Air 
Force  construction  efforts  and  would  produce  cost  growth  if  they  were  not  controlled. 

Jeljeli  and  Russell  (1995)  used  industry  surveys  and  SMEs  to  establish  41  critical 
risk  factors  and  probabilities,  which  were  rolled  up  into  the  six  broad  categories  of  Risk 
of  Technology,  Qualification  of  the  Contractor,  Effectiveness  of  Design,  Designer 
Management  Expertise,  Qualification  of  Designer,  and  Qualification  of  the  Owner.  Their 
study  identified  risk  factors  related  to  environmental  projects,  which  are  closely  related  to 
building  construction.  This  research  attempted  to  use  similar  techniques  to  estimate  the 
construction  cost  growth  or  required  contingency  amount  based  on  risk.  Similar  risk 
categories  are  involved,  with  two  primary  differences.  Since  technology  was  specifically 
related  to  remediation  techniques  and  sub-surface  contamination,  the  respective  factors 
were  not  applicable.  Qualifications  of  the  owner  are  unique  with  respect  to  this  research 
as  the  owner  is  considered  to  be  the  AF;  thus,  aspects  of  reputation  and  financial  stability 
were  not  considered.  After  discounting  technology  and  owner  qualifications,  4  categories 
with  15  factors  remained.  The  factors  considered  due  to  their  influence  on  construction 
cost  growth  included  contractor  experience,  contractor  past  performance,  contractor 
management  expertise,  constructability  of  design,  designer  experience,  and  designer  past 
performance;  the  category  of  Designer  Management  Expertise  (which  did  not  include 
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factors)  was  also  considered  relevant.  Accuracy  of  design  was  also  applicable  but  was 
identical  to  a  factor  used  from  Mustafa  and  Al-Bahar  (1991). 

This  research  also  included  additional  factors  for  the  SME  to  consider  based  on 
inputs  they  provided.  Drawing  on  the  research  applicable  to  new  construction  and 
renovation  projects,  four  factors  from  Mustafa  and  Al-Bahar  (1991)  and  seven  factors 
from  Jeljeli  and  Russell  (1995)  established  the  1 1  initial  risk  factors.  In  addition,  four 
other  factors  were  presented  to  the  SMEs  for  consideration:  age  of  facility,  asbestos, 
duration,  and  proximity  of  designer  to  site.  Age  and  asbestos  were  added  as  specific 
indicators  related  to  the  conditions  of  existing  facilities  to  be  renovated.  Duration  and 
proximity  were  added  as  possible  influences.  Together,  these  15  factors  were  a  first  step 
in  the  problem  of  quantifying  risk.  These  factors,  and  the  four  categories  within  which 
they  fall,  are  shown  in  Table  4.1.  These  risk  factors  established  the  starting  point  from 
which  to  proceed  to  phase  three. 
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Table  4. 1  Initial  Risk  Factors 


Factors 

Sub-Factors 

Physical 

Age  of  Facility 

Weather 

Asbestos 

Sub-Surface 

Contractor 

Experience 

Past  Performance 

Management  Experience 

Design 

Accuracy 

Constructability 

Duration 

Changes 

Designer 

Experience 

Past  Perfonnance 

Management  Expertise 

Proximity 

Phase  Two:  Through  the  literature  review  conducted  in  this  research,  a  composite 
list  of  15  risk  factors  was  developed.  The  individuals,  although  co-located,  were  queried 
separately  and  asked  to  provide  individual  responses.  Querying  those  individuals 
separately  avoids  the  possibility  of  “Group  Think”  which  is  the  tendency  of  one  or  more 
dominant  personalities  to  override  and  sway  consensus  to  their  views  (Clayton,  1997). 
The  SMEs  were  asked  to  give  their  opinion  on  the  relevance  and  applicability  of  the 
listed  risk  factors  based  on  their  knowledge  of  and  experience  with  AF  construction 
projects.  In  addition,  the  SMEs  were  given  the  opportunity  to  add  to  the  list  of  risk 
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factors  if  they  felt  the  list  was  not  complete.  The  SMEs  initially  added  four  risk  factors  to 
the  list  (see  the  “Interim  List”  column  in  Table  4.2).  After  several  iterations  in  which  the 
definitions  and  descriptions  of  the  factors  were  further  refined,  the  final  list  was 
comprised  of  the  13  factors  shown  in  the  “Final  List”  column  of  the  table. 
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Table  4.2  Risk  Factor  Development 


Initial  List 

Interim  List  Final  List 

1.  Age  of  Facility 

1.  Age  of  Facility 

1.  Age  of  Facility 

2.  Weather 

2.  Weather 

2.  Weather 

3.  Asbestos 

3.  Asbestos 

3.  Asbestos 

4.  Sub-Surface  (Ground  and 
interior  for  renovation) 

4.  Sub-Surface  (Ground  and 
interior  for  renovation) 

4.  Sub-Surface  (Ground 
and  interior  for 

renovation  or 
Differing/unforeseen 
site  conditions) 

5.  Contractor  Experience 

5.  Contractor  Experience 

6.  Contractor  Past 

Perfonnance 

6.  Contractor  Past 

Perfonnance 

7.  Contractor  Management 
Experience 

7.  Contractor  Management 
Experience 

5.  Contractor  Experience 

6.  Contractor  Past 

Performance 

8.  Designer  Experience 

8.  Designer  Experience 

9.  Designer  Past 

Performance 

9.  Designer  Past 

Performance 

7.  Contractor 

Management 

Experience 

10.  Proximity  of  Designer  to 

construction  location 

10.  Proximity  of  Designer  to 

construction  location 

8.  Designer  Experience 

1 1 .  Designer  Management 
Expertise 

1 1 .  Designer  Management 
Expertise 

9.  Designer  Past 

Performance 

12.  Design  Accuracy 

12.  Design  Accuracy 

10.  Design  Accuracy 

13.  Design  Constructability 

13.  Design  Constructability 

1 1 .  Design 

Constructability 

14.  Project  Duration 

14.  Project  Duration 

15.  Opportunity  for  Design 
Changes 

15.  Opportunity  for  Design 
Changes 

12.  Project  Duration 

13.  Opportunity  for 

Design  Changes 

16.  Type  of  Facility  to  be 

constructed 

17.  Users  understanding  of 
design  before 

construction 

18.  Differing/unforeseen  site 

conditions 

19.  Changes  in  user 
requirements 
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Phase  Three:  The  third  phase  was  the  development  of  the  likelihood  of 
occurrence  and  impact  of  each  of  the  final  risk  factors.  The  SMEs  were  asked  to  consider 
100  projects  and,  using  their  knowledge  and  experience,  categorize  each  risk  factor  into 
three  levels  of  risk.  The  definition  for  each  risk  factor  varied  but  was  briefly  categorized 
for  the  SME.  For  example,  weather  was  defined  as  being  mild,  normal,  and  severe.  The 
definitions  provided  to  the  SMEs  are  shown  in  the  range  column  of  the  table  in  Appendix 
C.  The  concept  was  for  the  SMEs  to  identify  the  number  of  projects  out  of  100 
associated  with  each  level  of  risk  for  each  risk  factor.  For  instance,  consider  Contractor 
Past  Perfonnance.  The  SME  may  have  thought  that  30  projects  had  low  or  poor 
Contractor  Past  Performance,  60  experienced  a  normal  level  of  perfonnance,  and  10  had 
contractors  with  exceptional  or  high  past  performance.  Phase  two  also  asked  the  SMEs 
to  rate  the  cost  impact  of  experiencing  the  specified  level  of  risk  for  each  factor  using  a 
standard  project  cost  of  $  1  million.  Continuing  with  the  Contractor  Past  Performance 
factor,  the  SME  may  have  thought  that  a  contractor  with  poor  past  perfonnance  might 
cost  the  government  an  additional  $100,000,  which  would  equate  to  a  10%  cost  growth. 
Additionally,  a  contractor  with  nonnal  past  performance  might  experience  a  $50,000  cost 
growth,  while  the  exceptional  performer  would  encounter  zero  cost  growth.  The  limiting 
factor  of  this  step  was  that  only  three  of  the  five  SMEs  responded. 

Step  Four 

Step  four  is  the  consolidation  and  summary  of  the  responses  acquired  from  the 
previous  Delphi  method  steps.  The  data  developed  from  the  Delphi  method  is  provided 
in  Appendix  C.  The  first  column  lists  the  risk  factors  developed  in  step  three  from  the 
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five  SMEs.  Column  two  contains  descriptions  of  the  risk  levels,  with  column  three 
providing  a  definition  in  the  range  column.  The  next  six  columns  represent  the 
probability  and  cost  impact  provided  by  the  three  SMEs.  The  data  shown  the  last  three 
columns  was  consolidated  through  statistical  averaging  to  formulate  a  consensus  of  the 
SMEs  estimates  for  Risk  Factor  likelihood  and  cost  impact.  Appendix  D  represents  step 
four  of  the  Delphi  process  and  thus  the  end  of  this  portion  of  the  research. 

Conclusion  of  the  Delphi  Method 

With  the  Delphi  portion  of  the  research  complete,  the  data  acquired  will  be  used 
to  construct  a  predictive  decision  model.  The  model  will  be  built  using  the  decision  tree 
method  in  an  effort  to  expand  each  risk  factor  through  each  level  of  risk  to  detennine  an 
expected  value.  The  expected  value  returned  by  a  decision  tree  model  will  be  the 
expected  cost  growth  for  a  project.  Therefore,  it  will  be  representative  of  the  amount  of 
contingency  funds  required  cover  unforeseen  events  associated  with  a  construction 
project. 

Decision  Tree 

The  decision  tree  model  is  fonnulated  through  a  progressive  expansion  of  each 
risk  factor  and  its  risk  levels.  The  expected  value  is  detennined  through  a  process  of 
rolling  back  each  branch  in  the  tree  (Jensen,  2001).  For  an  AF  construction  project,  a 
decision  tree  would  yield  a  dollar  value  representing  cost  growth  based  on  the  probability 
distribution  of  the  risks.  The  result  is  a  distribution  of  the  expected  cost  growth  and  the 
associated  probability  of  that  outcome.  The  expected  value  result  is  a  quantification  of 
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the  cost  growth  that  the  decision  maker  can  use  to  decide  whether  or  not  to  use  EVM. 
Using  the  13  risk  factors  and  their  three  levels  of  risk  developed  through  the  Delphi 
method,  the  resulting  decision  tree  model  has  a  final  expansion  of  1,594,323  branches. 
The  development  of  the  model  was  initially  constructed  using  DPL  4.0,  which  is  a 
professional  decision  analysis  software.  However,  the  computing  time  using  this 
software  was  prohibitive.  Therefore,  the  model  was  reconstructed  by  programming  a 
routine  in  Microsoft  Excel  using  Visual  Basic  Editor.  Each  process  will  be  explained  in 
order  to  discuss  the  results  of  the  model. 

DPL  4.0 

DPL  4.0  was  initially  selected  because  of  its  ability  to  convert  influence  diagrams 
into  decision  trees.  The  graphical  nature  shows  the  relationship  between  a  decision  and 
the  multitude  of  influences  reflected  in  the  decision.  Therefore,  the  influence  diagram 
shows  what  factors  effect  the  cost  growth  of  a  project.  The  amount  of  cost  growth  helps 
the  decision  maker  determine  the  applicability  of  EVM.  The  initial  iteration  of  the  model 
is  shown  as  an  influence  diagram  in  Figure  4.1.  The  subsequent  decision  tree  model  is 
shown  in  an  abbreviated  method  (i.e.,  not  fully  expanded)  in  Figure  4.2.  Due  to  the  size 
of  the  decision  tree,  the  model  execution  was  incomplete  after  72  hours  of  computing 
time.  The  failure  seemed  to  be  the  result  of  insufficient  computing  power  and  memory 
capacity  of  DPL  to  handle  a  decision  tree  of  this  size:  13  risk  factors  evaluated  at  three 
risk-levels  resulting  in  almost  1.6  million  branches  or  possible  outcomes.  As  a  result, 
another  method  for  developing  and  solving  the  model  had  to  be  found. 


64 


Figure  4.1  DPL  Initial  Influence  Diagram 
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V 
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Figure  4.2  DPL  Initial  Decision  Tree-Unexpanded 
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Microsoft  Excel  Visual  Basic 


The  next  iteration  of  the  model  was  programmed  in  Microsoft  Excel  using  the 
Visual  Basic  Editor.  Using  this  programming  language,  the  13  risk  factors  were  assigned 
a  two-dimensional  value  to  capture  the  likelihood  of  occurrence  and  the  cost  impact 
associated  with  each  level.  The  process  to  explore  the  nearly  1.6  million  possible 
outcomes  was  accomplished  through  a  series  of  imbedded  loops.  Each  loop  is  repeated 
for  each  risk  factor  and  again  for  each  risk  level.  The  calculation  of  the  expected  value 
was  performed  through  a  progressive  summation  of  each  of  the  1.6  million  iterations. 

The  entire  program  is  shown  in  Appendix  D.  The  input  data  for  the  program  is  gathered 
with  a  template  using  an  Excel  spreadsheet,  as  shown  in  Figure  4.3. 

To  test  both  the  spreadsheet  and  the  model,  the  collective  input  from  the  SMEs 
was  used.  Based  on  a  project  estimate  of  $  10  million,  the  results  of  the  initial  run  of  the 
decision  tree  resulted  in  an  expected  cost  growth  of  $5,958,851.22,  which  represents  a 
59.6%  cost  growth  (discussion  is  covered  in  the  analysis  section).  The  distribution  of  the 
4772  distinct  outcomes  is  shown  graphically  in  Figure  4.4.  In  addition  to  the  distribution 
of  the  possible  outcomes  and  their  associated  probabilities,  the  model  also  outputs  the 
cumulative  probability  of  the  possible  outcomes.  This  is  shown  in  Figure  4.5. 
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Risk-Based  Decision  Model 

Original  Cost  Estimate  =  $  10,000,000 

I  Age 

Weather 

Asbestos 

Probability 
%  Cost  Growth 

New 

Nominal 

Old 

None 

Nominal 

Severe 

None 

Minimal 

High 

0.6166 

0.1667 

0.2167 

0.2333 

0.5 

0.2667 

0.35 

0.35 

0.3 

0.00500 

0.04500 

0.16667 

0.00000 

0.02000 

0.06333 

0.00500 

0.05667 

0.12667 

Sub-Surface 

Contractor  Experience 

Contr  Past  Performance 

Probability 
%  Cost  Growth 

None 

Minimal 

High 

Little 

Moderate 

Significant 

Good 

Very  Good 

Exceptional 

0.5 

0.2833 

0.2167 

0.1667 

0.5333 

0.3 

0.2167 

0.6333 

0.15 

0.01167 

0.05667 

0.13333 

0.04667 

0.01633 

0.00000 

0.09333 

0.03000 

0.00000 

|  Contr  Management  Experience 

Design  Accuracy 

Constructability 

Probability 
%  Cost  Growth 

Low 

Nominal 

High 

Low 

Nominal 

High 

Easy 

Moderate 

Difficult 

0.1833 

0.5667 

0.25 

0.5333 

0.35 

0.1167 

0.2833 

0.5167 

0.2 

0.07500 

0.02833 

0.00000 

0.15000 

0.02833 

0.00000 

0.00000 

0.02000 

0.08667 

Duration 

Changes 

Designer  Experience 

Probability 
%  Cost  Growth 

<8M 

8-1 8M 

>18M 

None 

Minimal 

Significant 

Little 

Moderate 

Significant 

0.1333 

0.4167 

0.45 

0.1667 

0.5 

0.3333 

0.1833 

0.6 

0.2167 

0.01167 

0.03167 

0.06833 

0.04667 

0.07167 

0.10667 

0.12000 

0.01833 

0.00000 

Designer  Past  Performance 

Good 

Very  Good 

Exceptional 

0.25 

0.6 

0.15 

Probability 

0.11667 

0.03667 

0.00000 

%  Cost  Growth 

Figure  4.3  Risk  Factor  Input  Sheet 
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Figure  4.5  Initial  Model  Results  -  Cumulative  Probability 


Analysis 

With  the  model  fonnulated  and  running,  the  results  of  the  output  must  then  be 
examined.  This  analysis  will  be  conducted  in  two  phases.  The  first  phase  will  look  at  the 
results  of  the  initial  model  and  the  nature  of  the  cost  growth.  The  second  phase  will 
attempt  to  refine  the  model  and  produce  an  interactive  version  that  can  be  used  by  a 
project  manager  to  assess  his/her  specific  project. 
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Phase  One 


The  results  of  the  initial  model’s  output  predicted  a  cost  growth  of  nearly  60%. 

To  understand  the  nature  and  size  of  this  result,  the  individual  data  from  each  SME  was 
loaded  independently  into  the  model.  The  SME’s  data  returned  cost  growth  estimates 
averaging  approximately  60%;  the  values  were  $4,902  million  for  SME  A,  $6,675 
million  for  SME  B,  and  $6,320  million  for  SME  C  on  a  $10  million  project.  Using  the 
best-case  scenario  for  each  risk  factor  yielded  a  minimum  cost  growth  of  8%.  This  was 
due  to  the  fact  that  5  of  the  13  factors  did  not  have  a  zero  cost  impact  option.  At  the  other 
end  of  the  spectrum  (i.e.,  the  worst-case  scenario),  the  model  returned  a  cost  growth  of 
nearly  124%. 

These  results,  based  on  the  SME  data,  did  not  yield  realistic  cost  growth 
predictions.  The  results  indicate  that  the  SMEs  did  not  differentiate  between  a  single  risk 
factor  and  its  effect  on  other  risk  factors.  For  instance,  the  cost  growth  provided  by  the 
SMEs  for  the  effect  of  a  designer  with  a  “good”  past  performance  was  1 1 .7%.  This  is 
possible  if  the  SMEs  took  into  consideration  the  consequential  effect  on  other  factors 
such  as  changes,  unforeseen  site  conditions,  and  duration  rather  than  considering  those 
independent  of  designer  past  performance.  The  cause  of  this  error  is  a  result  of  the 
explanation  of  the  process  and  the  SME’s  inability  to  limit  the  cause  of  the  effect.  This  is 
consistent  with  previous  research  concluding  that  a  decision  maker  may  not  be  able  to 
disassociate  individual  parts  when  their  experience  deals  with  the  system  as  a  whole 
(Weir,  2005). 
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Phase  Two 


In  an  effort  to  create  a  model  which  would  allow  the  project  manager  to  input 
specific  information  for  any  given  project  under  evaluation,  the  Excel  spreadsheet  was 
modified  and  the  Visual  Basic  code  adjusted.  The  code  was  written  to  analyze  each  cell 
of  the  input  sheet  such  that  the  PM  could  designate  the  number  of  risk  levels, 
probabilities,  and  impact.  If  nothing  were  changed,  the  program  would  default  to  the 
values  assigned  by  the  SMEs.  Therefore,  a  project  manager  can  overwrite  the  data  with 
specific  information  and  details  based  on  their  particular  knowledge  of  the  project  in 
question.  Using  this  input,  the  model  can  calculate  the  specific  probability  distribution 
and  expected  cost  growth  factor  for  that  individual  project.  New  construction  for 
example  would  have  no  impact  due  to  asbestos  as  opposed  to  a  1960s  era  building  being 
completely  renovated.  The  program  recognizes  whether  the  user  has  input  a  one,  two,  or 
three  tier  distribution  risk  for  each  factor.  Again,  using  the  information  provided,  the 
model  runs  multiple  iterative  loops  in  which  it  calculates,  and  stores  the  thousands  of 
possible  cost  growths.  The  result  is  a  probability  distribution  representing  the  sum  of  all 
the  cost  growth  factors. 

In  addition,  the  age  and  duration  factors  were  adjusted.  In  the  previous  version, 
the  age  of  the  facility  and  estimated  project  duration  were  unknowns.  However,  these 
will  be  known  entities  for  the  PM.  Therefore,  the  probability  of  these  two  risk  events  was 
set  to  one  and  the  input  sheet  was  modified  to  allow  the  age  and  duration  to  be  directly 
input  by  the  PM.  To  be  consistent  with  the  Delphi  method,  “IF  THEN”  statements  were 
added  to  the  code  to  set  the  cost  impact  for  the  age  and  duration  to  correspond  to  the 
values  determined  by  the  SMEs  for  the  appropriate  level.  The  code  for  the  revised 
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program  is  shown  in  Appendix  E.  For  example,  if  the  input  age  for  a  facility  was 
between  15  and  30  years  old,  the  impact  was  assigned  a  4.5%  cost  growth.  An  example 
of  the  revised  input  sheet  is  shown  in  Figure  4.6. 


Risk-Based  Decision  Model 
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Figure  4.6  User  Input  Sheet 
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Summary 

The  result  of  the  user  input  model  is  a  tool  which  will  use  the  specific  information 
provided  on  the  applicable  project  to  return  an  expected  cost  growth.  The  decision  maker 
can  then  use  this  information  to  make  a  more  informed  choice  as  to  whether  EVM  is 
appropriate  for  that  particular  project.  This  meets  the  requirement  set  forth  by  the  revised 
DoD  EVM  policy  letter  for  a  “Risk-Based  Decision.” 
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CHAPTER  5 


CONCLUSIONS 


The  purpose  of  this  research  was  to  develop  a  model  which  would  quantify  the 
risks  associated  with  a  typical  construction  project  in  an  effort  to  meet  revised 
requirements  set  forth  by  the  Department  of  Defense  (DoD)  regarding  the  use  of  earned 
value  management  (EVM).  The  revised  policy  stated  that  the  use  of  EVM  for  projects 
under  $20  million  would  be  a  “risk-based  decision.”  Therefore,  a  spreadsheet-based 
model  was  created  to  evaluate  a  project  manager’s  subjective  and  appropriate  knowledge 
of  risk  factors  concerning  a  specific  project.  The  output  of  the  model  is  the  expected  cost 
growth  associated  with  the  project.  Since  the  cost  growth  is  based  on  risk  factors,  the 
model  can  be  used  to  determine  whether  the  additional  cost  of  implementing  an  EVM 
system  is  justified.  The  chapter  presents  the  results  of  this  research  by  addressing  each  of 
the  original  research  questions.  In  addition,  information  will  be  discussed  concerning 
recommendations,  strengths  and  weaknesses,  and  possible  areas  of  future  research. 

Question  1 

The  first  research  question  asked:  What  risk  factors  affect  the  cost  or  schedule  of 
Air  Force  construction  projects?  The  results  of  the  literature  review  revealed  that  many 
factors  affect  the  outcome  of  a  construction  project.  Through  the  use  of  the  Delphi 
method  with  subject  matter  experts  (SMEs),  this  research  identified  13  risk  factors  that 
were  considered  the  most  relevant  and  applicable  to  the  cost  growth  experienced  during 
construction  of  Air  Force  (AF)  projects.  Table  5.1  summarizes  these  factors. 
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Table  5 . 1  Construction  Risk  F actors 


Factors 

Sub-Factors 

Physical 

Age  of  Facility 

Weather 

Asbestos 

Sub-Surface  Conditions 

Contractor 

Experience 

Past  Performance 

Management  Experience 

Design 

Accuracy 

Constructability 

Duration 

Changes 

Designer 

Experience 

Past  Performance 

Question  2 

The  second  research  question  asked:  What  is  the  probability  of  occurrence  of  the 
identified  risk  factors?  To  answer  this  question,  the  Delphi  method  was  used  with  the 
SMEs  to  survey  their  collective  knowledge  and  experience;  consequently,  a  three-tier  risk 
profile  (low,  medium,  and  high)  was  developed  for  each  identified  risk  factor.  The 
results  provided  valuable  insight  into  the  likelihood  of  experiencing  a  range  of  each  risk 
factor  from  a  minimal  impact,  through  an  expected  or  moderate  impact,  to  an  extreme  or 
high  impact.  In  addition,  the  research  revealed  that  two  of  the  risk  factors  (age  of  facility 
and  duration  of  the  project)  would  be  known  entities.  Therefore,  the  specific  information 
could  be  provided  by  the  project  manager  with  100%  certainty. 
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Question  3 

The  third  research  question  asked:  What  is  the  subsequent  cost  growth  due  to  the 
occurrence  of  the  risk  factor?  With  the  range  of  risk  levels  identified  from  question  two, 
the  SMEs  were  asked  to  use  their  knowledge  and  experience  to  estimate  the  incremental 
cost  growth  associated  with  each  risk  factor  for  the  three  levels  of  risk.  The  SMEs  were 
asked  to  look  at  each  level  of  each  risk  factor  and  estimate  an  associated  cost  growth 
independent  of  all  other  factors.  However,  the  SMEs  had  difficulty  compartmentalizing 
the  effects  of  individual  risk  factors;  they  indicated  that  the  impact  of  a  given  risk  factor 
was  a  summation  of  the  impact  across  the  entire  project  including  the  effect  on  other 
factors.  The  results,  therefore,  are  biased  by  approximately  ten  percent.  The  correction 
to  this  inaccuracy  is  the  ability  of  the  model  to  use  project  manager  input  to  recalculate 
the  expected  cost  growth  from  information  specific  to  the  project  in  question. 

Question  4 

The  fourth  research  question  asked:  How  many  AF  Construction  projects  in  the 
last  5-years  and  those  programmed  through  the  FY12  exceed  the  $20M  threshold  or  fall 
into  a  range  which  must  be  evaluated?  From  infonnation  provided  by  Cook  (2006),  the 
following  project  counts  were  accurate  as  of  15  May  2005.  Of  3 14  military  construction 
projects  in  the  Automated  Civil  Engineer  System  database,  125  projects  are  between  $5 
million  and  $20  million  and  7  Projects  are  over  the  $20  million  threshold. 
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Recommendations 


The  recommendation  of  this  research  is  to  use  the  decision  tree  and  model 
developed  from  it  to  provide  a  quantitative  analysis  of  risk,  which  can  then  be  used  to 
help  detennine  the  applicability  of  using  EVM  on  construction  projects.  The  decision  to 
use  EVM,  therefore,  implies  that  sufficient  risk  is  present  to  warrant  the  cost  of  EVM. 
Due  to  the  cost  of  implementing  EVM,  this  model  should  probably  be  used  only  on  Air 
Force  construction  projects  with  cost  estimates  over  $5  million.  If  the  cost  growth 
predicted  by  the  model  is  greater  than  5%,  the  use  of  EVM  is  recommended  to  monitor 
the  risk  factors.  However,  this  recommendation  should  be  tempered  with  the  overall  risk 
associated  with  a  given  project.  In  other  words,  if  the  calculated  cost  growth  is  high  but 
the  probability  of  occurrence  is  low,  the  decision-maker  may  want  to  forego  the  use  of 
EVM  and  the  associated  costs. 

An  additional  recommendation  concerns  the  model  development.  Instead  of 
relying  on  spreadsheet-based  calculations,  the  use  of  a  commercial  off-the-shelf  decision 
analysis  software  package  that  has  the  capacity  to  calculate  a  large-scale  decision  tree 
would  be  advantageous.  A  program  which  offers  built-in  sensitivity  analysis  and  can 
handle  multiple  risk  levels  would  benefit  the  decision  maker  greatly. 

Strengths  and  Weaknesses 

The  primary  strength  of  the  model  developed  in  this  research  is  quantification  of 
subjective  risk  factors.  The  use  of  the  Delphi  method  provided  the  opportunity  to 
develop  a  consensus  regarding  appropriate  risk  factors  and  the  levels  of  risk  associated 
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with  each  factor.  Another  strength  is  the  ability  of  individual  project  managers  to  input 
project-specific  data  based  on  their  knowledge  and  experience. 

Several  factors  were  seen  as  weaknesses  impacting  this  research.  The  primary 
concern  was  the  development  of  relevant  data  related  to  the  risk  factors.  The  Delphi 
approach  was  appropriate  but  the  process  requires  further  refinement.  The  probabilities 
of  the  risk  event  levels  and  the  associated  cost  impacts  received  from  the  SMEs  were  not 
in  line  with  realistic  outcomes.  The  process  was  inhibited  by  a  lack  of  face-to-face 
interaction,  the  scheduling  of  time  by  the  SMEs  due  to  busy  schedules  and  temporary 
duty,  the  inherent  difficulties  related  to  a  reliance  on  e-mail,  and  the  difficulty  in 
compartmentalizing  one  risk  impact  independent  of  the  others.  The  most  significant  of 
these  sources  of  error  was  the  compartmentalization  of  the  risk  impact. 

Subsequent  analysis  showed  that  the  responses  of  the  SMEs  regarding  risk  impact 
were  actually  a  cumulative  cost  impact  of  all  risk  factors.  In  hindsight,  the  SMEs  should 
have  been  asked  to  consider  all  other  factors,  besides  the  risk  factor  in  question,  to 
represent  the  best-case  scenario;  in  this  way,  the  incremental  cost  increase  would  be  due 
only  to  the  specific  risk  at  the  specified  level.  Part  of  the  problem  may  have  been  caused 
by  inadequate  definitions  of  the  risk  factors  and  their  respective  risk  levels.  The  validity 
of  the  model  could  be  increased  through  an  extensive  survey  of  a  broad  range  of  SMEs 
and  through  successive  phases  to  narrow  the  definitions  and  assumptions.  The  risk 
factors  themselves  could  be  in  question;  although  validated  by  the  five  SMEs,  the  list  is 
neither  exhaustive  nor  complete. 
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Areas  of  Future  Research 


An  area  of  future  research  involves  the  use  of  a  formal  survey  to  query  the  Civil 
Engineering  career  field.  The  survey  could  establish  relevant  risk  factors  and  provide  a 
more  extensive  knowledge  base  for  the  establishment  of  accurate  probabilities  and  cost 
growth.  Utilizing  the  Delphi  methodology  with  a  large  pool  of  experts  and  multiple 
iterations  to  refine  the  probabilities  and  impact  would  serve  to  improve  the  model  and 
help  future  decision  makers. 
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Appendix  A 


™t  limqilf*  secseTAfiy  op  dEperjse 

KilODEFEN&E  PENTAGON 
WASHINGTON.  P.C  HBCKI  30!  0 


ibp  7  m, 


MEMORANDUM  FOR:  SEE  DISTRIBUTION 

SirHJF.CT:  Revision  to  I>oD  Earned  Value  Msuageihtttt  Policy 

Earnad  V a Luc  M eci e g tii'i c nt  (EVM)  bos  been  an  effective  naeintpimwU  control  Ic-cl 
in  ttlt  Department  for  the  pust  5?  years.  In  Orderto  Htrcamlute,  improve,  and  increase 
consistency  in  EVM  irnplwoenlatiw  And  application.  I  bid  revising  Lilt  policy  tfl  jnclLudfi 
the  following,  changes,  effective  immediately. 

L  Cost  of  incentive  contracts,  sibcontraolls,  inp^go^'flllintnlworlt  agreemsnts.  and 
A(htt  agreements  valued  al  otr  jyealer'  than  E20  million  in  Lhen-ytiM'  dsll&fs  shall 
implement  the  American  National  Standards  InslitutertilettKmic  Industries 
Alliance  Standard,  748.  Earned  Vaiut  ManagfPWJt  Systems  (ANSfi'EIA-ldE), 

Cofll  or  incentive  con  tracts.  subconiraclS,  Hfd  other  agreement!;  val  ued  aJ  or 
greats  ihm  550-  mil  lion  ir.  thtn-yesr  dollars  shaJt  have  an  EVM  system  that  ha:- 
hei*n  formally  validated  and  accepted  by  ih-r  cojniiiuil  COftincting  officer.  I 
intend  to  review  these  dollar  tfneaholdst  and  revise  them  if  necessary,  as  l*aae 
every  five  yeais. 

2.  A  Control  Pesfontnaflcc  Report  fCPR}<D«ta  [ton  Description  (DID)  number  DT- 
MtrMT-SUS*)  (previously  called  she  Cost  Pefomiance  Report)  Mid  Aft  Integrated 
Master  Schedule  (IMS)  (DID  number  pl-MGMT-SlbiO)  shall  be  required 
whenever  EVM  (eompItMH#  with  ANSl/E[A-748>  b  required,  that  is.  far  cost  0* 
incentive  cbrtDrftCts,  subcantrads.  infra -government  wort  aigrec  raen.13,  and  <rther 
agrefTnewts  valued  at  or  greaur  than  E20  million  in  ihen-yeaj  dflllnfi.  However, 
CPU  and  IMS  reporting  for  rotor  incentive  conlraCTS,  subeontrads,  intra- 
gnvemnienl  work  agreements,  and  other  agreements  valued  at  less  Lhtui  ESQ 
million  in  then-year  doUers  may  be  tailored  (refer  to  die  DoD  Earned  Ve.||*±j 
Mnmlgeipient  1  mplemcnulion  Guide  {EVMIG)  for  guidance  on  (a.ilnrlng 
reporting).  A  common  work  breakdown  structure  that  follows  thr  DoD  Work 
Breakdown  Structure  Handbook  (MIL-HDBK'fSlj  shail  he  used  for  theCPR, 
IMS,  md  Contractor  Cost  Data  Report  (CCDRJ.  The  CutfSdwIUfe  Status  Report 
(C/SSH)  is  rescinded  effective  immediately  (except  lb  to  extent  chat  it  is  required 
under  current  conlrWLS)  and  shall  not  boused  to  SMisiy  the  EVM  reporting, 
requirement  on  future  contract  awards 
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3.  Integrated  Baseline  Reviews  (IBRs)  shall  be  required  whenever  EVM  (compliance 
with  ANSI/E  1A-748)  is  required,  that  is,  for  cost  or  incentive  contracts, 
subcontracts,  intra-government  work  agreements,  and  other  agreements  valued  at 
or  greater  than  S20  million  in  then-year  dollars. 

4.  The  responsibility  and  requirement  for  government  surveillance  of  contracts 
remains  unchanged  and  shall  be  based  on  the  effectiveness  of  the  contractor’s 
implementation  of  internal  management  controls.  Guidance  on  surveillance 
activity  can  be  found  in  the  DoD  EVMIG. 

5.  EVM  is  discouraged  on  firm-fixed  price,  level  of  effort,  and  time  and  materials 
efforts,  including  contracts,  subcontracts,  intra-government  work  agreements,  and 
other  agreements,  regardless  of  dollar  value.  If  knowledge  by  both  parties  requires 
access  to  cost/schcdulc  data,  the  first  action  is  to  re-examine  the  contract  type 
(e.g.,  fixed  price  incentive).  However,  in  extraordinary  cases  where  cost/schedule 
visibility  is  required  and  cannot  be  obtained  using  the  Truth  in  Negotiation  Act, 
the  program  manager  shall  obtain  a  waiver  for  individual  contracts  from  the 
Milestone  Decision  Authority.  In  these  cases  the  program  manager  will  conduct  a 
business  case  that  includes  rationale  for  why  a  cost  or  fixed  price  incentive 
contract  was  not  an  appropriate  contracting  vehicle. 

6.  The  application  of  EVM  on  cost  or  incentive  efforts,  including  contracts, 
subcontracts,  intra-government  work  agreements,  and  other  agreements  valued  at 
less  than  $20  million  is  optional  and  is  a  risk-based  decision  that  is  at  the 
discretion  of  the  program  manager.  A  cost-benefit  analysis  shall  be  conducted 
before  deciding  to  implement  EVM  in  these  situations.  Considerations  for 
determining  the  efficacy  of  applying  EVM  in  these  situations  and  guidance  for 
tailoring  reporting  can  be  found  in  the  DoD  EVMIG. 

These  changes  to  EVM  policy  are  not  retroactive  but  shall  be  implemented  on  all 
applicable  future  contracts  that  arc  awarded  based  on  solicitations  or  requests  for 
proposal  issued  on  or  after  30  days  from  the  date  of  this  memorandum.  These  changes 
will  be  included  in  the  next  revision  of  the  DoD  5000  scries  and  other  acquisition-related 
documents.  While  there  is  no  prohibition  on  negotiating  the  revised  policy  into  current 
contracts,  the  costs  associated  with  changing  the  EVM  requirements  on  existing  contracts 
shall  be  borne  by  the  government. 

In  support  of  the  above  policy  changes,  the  Director,  Acquisition  Resources  and 
Analysis,  shall  update  all  pertinent  documents,  to  include  DoD  Instruction  5000.2,  the 
Defense  Acquisition  Guidebook,  and  the  CPR  and  IMS  DIDs.  The  Director,  Acquisition 
Resources  and  Analysis,  shall  work  with  the  Director,  Defense  Procurement  and 
Acquisition  Policy,  to  update  the  Defense  Federal  Acquisition  Regulation  Supplement 
(DFARS)  clauses.  The  Defense  Contract  Management  Agency  shall  lead  the  efforts  to 
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update  the  DoD  EVM1G.  The  Defense  Acquisition  University  shall  update  its  EVM 
curriculum. 

Until  the  updated  DFARS  clauses  are  coordinated  and  approved,  the  existing 
clauses  (252,234-7000  for  solicitations  and  252.234-7001  for  contracts)  shall  be  used. 

For  contracts  valued  at  or  greater  than  $50  million,  these  clauses  shall  be  applied  directly; 
For  contracts  valued  at  or  greater  than  $20  million  but  less  than  $50  million,  the 
following  paragraph  shall  be  included  in  the  statement  of  work:  "In  regard  to  DFARS 
252.234-7000  and  252. 234-7001,  the  contractor  is  required  to  have  an  Earned  Value 
Management  System  that  complies  with  ANSI/EIA-748;  however,  the  government  will  not 
formally  validate/accept  the  contractor 's  management  system  (no  formal  review), " 

While  not  required,  if  a  risk-based  decision  is  made  to  require  EVM  on  cost  or  incentive 
contracts  valued  at  less  than  $20  million,  the  above  paragraph  shall  also  be  included  in 
the  statement  of  work. 

While  it  is  preferred  that  Project  Management/E amed  Value  Management  costs  be 
charged  direct  to  the  contract,  the  contractor  shall  follow  their  accounting  policies  and 
procedures. 

Questions  regarding  the  revised  EVM  policy  should  be  directed  to  Ms.  Debbie 
Tomsk  (deborah  tomsic@osd.mil)  or  Mr.  Larry  Axtell  narrv.axtell@osd.mil)  at  (703) 
695-0707. 


DISTRIBUTION: 

SECRETARIES  OF  THE  MILITARY  DEPARTMENTS 
CHAIRMAN  OF  THE  JOINT  CHIEFS  OF  STAFF 

ASSISTANT  SECRETARY  OF  DEFENSE  (NETWORKS  AND  INFORMATION 
INTEGRATION/CHIEF  INFORMATION  OFFICER) 

GENERAL  COUNSEL  OF  THE  DEPARTMENT  OF  DEFENSE 
COMMANDER,  SPECIAL  OPERATIONS  COMMAND 
DIRECTOR.  ADMINISTRATION  AND  MANAGEMENT 
DIRECTOR,  PROGRAM  ANALYSIS  AND  EVALUATION 
DIRECTOR,  OPERATIONAL  TEST  AND  EVALUATION 
DIRECTOR.  DEFENSE  ADVANCED  RESEARCH  PROJECTS  AGENCY 
DIRECTOR.  DEFENSE  CONTRACT  MANAGEMENT  AGENCY 
DIRECTOR.  DEFENSE  INTELLIGENCE  AGENCY 
DIRECTOR,  DEFENSE  LOGISTICS  AGENCY 
DIRECTOR,  DEFENSE  THREAT  REDUCTION  AGENCY 
DIRECTOR,  MISSILE  DEFENSE  AGENCY 
DIRECTOR,  NATIONAL  SECURTIY  AGENCY 
PRESIDENT,  DEFENSE  ACQUISITION  UNIVERSITY 

CC: 

DIRECTOR,  NATIONAL  GEOSPATIAL-INTELLIGENCE  AGENCY 
DIRECTOR,  NATIONAL  RECONNAISSANCE  OFFICE 
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Appendix  B 


PROPOSED  NEW  EARNED  VALUE  MANAGEMENT  POLICY 

AND  RATIONALE 

Fitzgerald’s  First  Law  of  Program  Management z1  “There  are  only  two  phases 
to  a  big  military  program:  Too  early  to  tell  and  too  late  to  stop.  Program 
advocates  like  to  keep  bad  news  covered  up  until  they  have  spent  so  much 
money  that  they  can  advance  the  sunk-cost  argument;  that  it’s  too  late  to 
cancel  the  program  because  we’ve  spent  too  much  already." 

Benefits  of  EVM 


Although  cynical,  the  above  axiom  is  all  too  real  and  common,  not  generally  because  of 
the  deliberate  withholding  or  misrepresentation  of  information,  but  because  of  a  simple 
lack  of  objective  and  accurate  infonnation  on  current  program  status  and  what  that 
translates  to  in  terms  of  most  likely  program  outcomes.  Effective  management  control 
is  lost  the  moment  accurate  information  on  where  a  program  is  and  where  it  will  end  up  is 
unavailable. 

This  is  simply  unacceptable.  Management  is  accountable  for  program  outcomes,  so  it 
follows  that  they  must  have  the  necessary  infonnation  that  will  allow  them  to  effectively 
control  their  programs  and  thus  fulfill  their  responsibility. 

If  implemented  correctly,  Earned  Value  Management  (EVM)  provides  the  necessary 
information  to  avoid  Fitzgerald’s  First  Law  of  Program  Management. 

This  is  because  EVM  is  an  integrating  project  management  tool  that  facilitates 
improved  planning  and  control  of  cost,  schedule,  and  work  scope.  It  integrates  the 
functional  stovepipes  of  work  scope,  schedule,  and  cost  to  create  an  aggregate  picture  of 
performance,  thus  providing  management  with  the  necessary  infonnation  to  ensure  that 
day-to-day  decisions  keep  performance  consistent  with  program  objectives. 

To  put  it  another  way,  EVM  provides  an  early  warning  system  for  deviations  from  plan 
and  quantifies  technical  and  schedule  problems  in  cost  terms,  providing  a  sound  and 
objective  basis  for  considering  corrective  actions  (i.e.,  work  arounds,  tradeoffs,  etc.). 
Thus,  EVM  both  mitigates  the  risk  of  cost  and  schedule  overruns,  while  also  providing  a 
forecast  of  final  cost  and  schedule  outcomes. 

The  following  recommendations  aim  to  simplify,  improve,  and  increase  consistency  in 
Department  of  Defense  (DoD)  EVM  policy  and  processes.  They  are  a  reflection  of 
modem  day  EVM  best  practices  aimed  at  bringing  EVM  within  DoD  into  the  21st 
Century. 
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EVM  Management  Standard  -  Industry  Standard 


Recommendation  1 :  A  contractor  Earned  Value  Management  System  (EVMS)  compliant 
with  the  current  version  of  American  National  Standards  Institute/Electronic  Industries 
Alliance  Standard  748  (ANSI/EIA-748) — a.k.a.  Industry  Standard  (as  interpreted  by  the 
Guidelines  Intent  document  for  complex  high-risk  projects)  will  be  required  whenever 
EVM  is  required. 


The  32  EVM  Guidelines  in  the  Industry  Standard  do  not  tell  the  contractor  how  to  do  the 
work,  instead  they  establish  minimum  management  control  guidelines  for  an  EVMS. 
Every  EVMS  must  comply  with  these  guidelines,  otherwise  there  is  no  assurance  that 
the  EVM  data  being  provided  and  relied  upon  by  management  is  valid.  The  Guidelines 
Intent  document  further  interprets  the  intent  of  these  guidelines. 


The  Cost/Schedule  Status  Report  (C/SSR)  and  Cost  Perfonnance  Report-No  Criteria  will 
be  eliminated  because  they  do  not  require  contractor  compliance  with  any  set  of 
minimum  management  control  principles.  Eliminating  these  categories  standardizes 
EVM  reporting  requirements  and  improves  the  timeliness,  accuracy,  and  reliability  of  the 
data  created  by  the  EVMS.  In  addition,  eliminating  these  categories  emphasises  that 
DoD  is  serious  about  doing  EVM  correctly  (i.e.,  against  a  verifiable  standard). 

EVM  Reporting 


Recommendation  2:  A  Contract  Performance  Report  (CPR)  and  Integrated  Master 
Schedule  (IMS)  will  be  required  whenever  EVM  is  required. 


EVM  reporting  provides  the  customer  program  office  insight  into  (i.e.,  not  oversight  of) 
contractor  performance.  This  is  vital  to  enable  the  customer  to  provide  the  specific  and 
timely  input  that  the  contractor  needs  in  order  to  succeed,  and  to  fulfill  the  DoD  program 
manager’s  monitoring  and  reporting  role  to  higher  management.  It  facilitates 
communication  between  contractor  and  customer  because  it  presents  an  accurate  picture 
of  project  perfonnance  at  whichever  level  of  the  Work  Breakdown  Structure  (WBS) 
(required)  and  Organizational  Breakdown  Structure  (OBS)  (optional)  is  being  reviewed. 
It  also  maximizes  project  management  efficiency  because  EV  data  allows  for 
management  by  exception  by  both  the  DoD  customer  and  the  contractor,  while  enabling 
the  contractor  team  to  manage  the  day-to-day  work  at  the  task  level. 
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The  32  EVM  Guidelines  in  the  Industry  Standard  do  not  cover  the  field  of  EV  data 
reporting  requirements.  It  is  left  to  the  customer  to  determine  the  details  of  the  EV  data 
to  be  reported  and  the  level  of  analysis  required.  As  a  consequence,  the  customer 
reporting  requirements  can  require  a  higher  standard  of  data  and  analysis.  A  higher 
minimum  standard  of  data  and  analysis  will  be  adopted  in  the  updated  Data  Item 
Descriptions  (DIDs)  to  reflect  industry  best  practice  and  enable  the  use  of  modern 
EVM  software  tools.  These  changes  are  aimed  at  ensuring  contractors  and  DoD 
program  offices  ‘use  EV  data  to  manage,’  rather  than  merely  trying  to  ‘manage  the  EV 
data.’  At  an  appropriate  time,  the  Industry  Standard  itself  may  need  to  be  updated,  as 
industry  best  practice  becomes  the  norm. 

The  most  significant  changes  to  the  updated  CPR  (to  be  called  Contract 

Performance  Report)  include: 

•  Reducing  the  time  period  within  which  EV  data  and  analysis  must  be 
available,  and  strengthening  the  requirement  for  digital  submission  - 
this  will  improve  the  timeliness  of  reporting,  hence  increasing  its 
management  value. 

•  Requiring  more  comprehensive  data  (e.g.,  separating  out  General  and 
Administrative  (G&A),  major  subcontractor,  material,  reprogramming 
adjustments,  and  Over  Target  Baselines;  and  requiring  Performance 
Measurement  Baseline  (PMB),  Estimate  at  Completion  (EAC),  and 
staffing  forecasts  monthly  for  the  next  six  months  and  monthly, 
quarterly,  or  annually  to  the  end  of  the  contract)  -  this  will  improve 
visibility  of  performance. 

•  Requiring  a  minimum  set  of  requirements  and  more  comprehensive 
analysis  in  Format  5  explanations  -  this  will  ensure  the  data  is  used, 
understood,  and  acted  upon. 

The  most  significant  changes  to  the  updated  IMS  include: 

•  Mandating  and  integrating  the  IMS  with  the  CPR  (e.g.,  identical  contract  WBS  to 
be  used,  concurrent  delivery,  concurrent  analysis)  -  sound  scheduling  with  critical 
path  analysis  has  always  been  a  key  part  of  EVM  (3  EVM  Guidelines  deal  with  it 
specifically),  yet  previous  policy  has  overlooked  the  IMS.  Mandating  and  integrating 
the  IMS  with  the  CPR  corrects  this  oversight. 

•  Requiring  a  fully  integrated  network  of  all  discrete  contract  tasks/activities  -  this 
will  improve  scheduling  and  eliminate  networking  of  only  ‘significant’ 
interdependencies,  an  approach  that  has  serious  shortcomings,  which  have  been  noted 
on  numerous  programs. 

As  a  consequence  of  eliminating  the  C/SSR  category,  C/SSR  reports  are  also  eliminated. 
The  CPR  will  remain  tailorable  and  tailoring  guidance  is  being  developed  for  inclusion  in 
the  Earned  Value  Management  Implementation  Guide  (EVMIG).  A  tailored  CPR  for 
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contracts  under  the  $50M  threshold  will  include  as  a  minimum  Format  1  and  a  tailored 
Format  5  (Formats  2,  3,  and  4  will  be  optional).  Within  the  parameter  of  requiring  a  fully 
integrated  network  of  all  discrete  tasks/activities,  the  IMS  will  also  be  tailorable  for 
contracts  under  the  $50M  threshold  and  tailoring  guidance  is  being  developed  for 
inclusion  in  the  EVMIG. 

EVM  Application  Threshold 


Recommendation  3:  EVM  compliant  with  the  Industry  Standard  will  be  required  on 
all  cost  or  incentive  contracts,  subcontracts,  intra-government  work  agreements,  and 
other  agreements  greater  than  or  equal  to  $20M  (optional  for  contracts  of  less  than  12 
months  in  duration  and  non-schedule  based  types  of  contracts  such  as  time  and 
materials,  level  of  effort,  etc.). 

Recommendation  4:  EVM  for  all  cost  or  incentive  contracts,  subcontracts,  intra- 
government  work  agreements,  and  other  agreements  below  $20M  will  be  a  risk-based 
decision  (unless  designated  as  a  “major  acquisition”  in  accordance  with  Office  of 
Management  and  Budget  (OMB)  Circular  A-l  1). 


As  noted  above,  EVM  is  a  risk  mitigator  of  cost  and  schedule  overruns. 
Consequently,  where  the  DoD  program  manager’s  risk  analysis  has  determined  that 
sufficient  risk  exists  to  justify  a  cost  or  incentive  contract,  subcontract,  or  other 
agreement,  then  there  is  also  sufficient  risk  to  require  EVM.  The  Research, 
Development,  Test,  and  Evaluation  (RDT&E)  versus  Procurement/Operations  and 
Maintenance  (O&M)  distinction  is  a  factor  that  is  taken  into  account  when  deciding 
contract  type  (i.e.,  risk),  so  it  is  redundant  to  again  make  it  a  factor  in  determining 
EVM  application. 

The  above  application  threshold  represents  a  raising  of  the  current  C/SSR  threshold 
(from  $6.3M)  and  a  lowering  of  the  current  CPR  thresholds  for  RDT&E  (from 
$73M)  and  Procurement/O&M  (from  $315M).  The  $20M  threshold  aligns  with  U.S. 
Department  of  Energy  practice  and  Australian  practice.  It  is  a  balanced  approach 
that  gives  greater  discretion  to  program  managers  at  lower  contract  values,  while 
also  recognizing  that  the  current  thresholds  are  outdated. 
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Recommendation  5:  EVM  for  all  firm  fixed  price  contracts,  subcontracts,  intra- 
government  work  agreements,  and  other  agreements  will  be  a  risk-based  decision. 


Firm  fixed  price  contracts  can  still  expose  the  customer  to  significant  schedule  risk  and 
even  residual  cost  risk  if  cost  overruns  cause  the  contractor  to  ‘cut  corners’  (e.g., 
reducing  quality  as  less  experienced,  lower  cost  staff  perform  the  work  and  inferior 
materials  are  used,  reducing  testing,  increasing  the  cost  of  contract  changes  and  follow-on 
support  contracts  to  recoup  the  overrun).  Consequently,  EVM  on  firm  fixed  price 
contracts  is  a  risk  mitigation  option  that  will  be  available  to  program  managers,  but  not 
mandated 

(Note:  Tailoring  guidance  for  the  CPR  and  IMS  on  firm  fixed  price  contracts, 
subcontracts,  intra-government  work  agreements,  and  other  agreements  will  be  provided 
in  the  EVMIG,  along  with  guidance  that  EVM  should  not  be  required  on  contracts  of  less 
than  12  months  in  duration  and  non-schedule  based  types  of  contracts  such  as  time  and 
materials,  level  of  effort,  etc.  Definitions  of  the  non-schedule  based  contract  types  will 
be  included  in  the  guidance.) 

EVM  Implementation 


Recommendation  6:  A  validation  of  the  contractor  E  VMS’s  compliance  with  the 
Industry  Standard  will  be  required  on  all  cost  or  incentive  contracts,  subcontracts, 
intra-government  work  agreements,  and  other  agreements  greater  than  or  equal  to 
$50M.  Validation  for  all  other  cost  or  incentive  contracts,  subcontracts,  intra- 
government  work  agreements,  and  other  agreements  and  for  any  firm  fixed  price 
contracts,  subcontracts,  intra-government  work  agreements,  and  other  agreements  for 
which  EVM  is  required  will  be  a  risk-based  decision. 

Once  a  contractor’s  EVMS  is  validated  and  formally  accepted  by  DoD,  acceptance  will 
continue  to  be  affirmed  via  government  surveillance.  Validation  does  impose  costs. 
These  costs  should  and  will  be  borne  by  the  government  for  the  following  reasons: 

•  The  primary  purpose  of  the  validation  is  to  mitigate  the  inherent  performance  risk  of 
any  supplier  that  undertakes  a  high-risk  contract.  This  is  fundamentally  a  customer 
responsibility  and  reflects  an  appropriate  investment  in  managing  taxpayer  funds 
efficiently  and  effectively.  This  responsibility  can  be  outsourced  to  third  parties 
(although  note  the  third  bullet  below),  but  contractors  must  not  be  allowed  to  self- 
regulate;  there  must  be  independent  review  of  the  contractor’s  processes  to  ensure 
the  integrity  of  the  validation. 

•  High-risk  contracts  impose  risk  on  the  customer  and  this  is  accepted  through  cost  or 
incentive  type  contracts,  subcontracts,  intra-government  work  agreements,  and  other 
agreements.  The  necessity  to  allocate  resources  to  manage  this  risk  is  self-evident 
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and  should  be  explicitly  acknowledged  and  funded  accordingly.  Funding  the 
validation  process  shares  some  of  the  costs  of  requiring  higher  standards  of  project 
management  with  contractors. 

•  DoD  validation  improves  DoD’s  ability  to  be  an  ‘informed  customer’  -  it  is 
important  on  high-risk  contracts  for  both  the  customer  and  contractor  to  have  higher 
levels  of  capability,  knowledge,  and  skills.  If  the  customer  is  not  informed,  then 
contractor  performance  is  adversely  affected. 

•  Validation  is  an  important  means  to  facilitate  continuous  improvement  because  it 
brings  the  combined  wisdom  of  a  team  of  experts  to  provide  a  reality  check  on  a 
contractor’s  EVMS.  A  balance  in  the  investment  required  of  the  customer  to 
undertake  this  role  can  be  achieved  by  not  requiring  validation  at  the  lower  value  end 
of  the  scale,  although  compliance  with  the  Industry  Standard  is  still  required  by  the 
contractor. 

Recommendation  7:  Government  surveillance  of  the  contractor’s  EVMS  will  be 
required  whenever  EVM  is  required. 

Government  surveillance  is  required,  regardless  of  whether  there  is  an  upfront  validation, 
to  provide  assurance  that  contractors  comply  with  their  contractual  obligations  and  to 
ensure  that  the  EV  data  being  provided  and  relied  upon  by  management  is  valid.  The 
customer  must  monitor  contractor  performance  in  order  to  fulfill  its  obligations  to  the 
public  in  managing  taxpayer  funds.  The  degree  of  government  surveillance  should  be 
based  on  the  effectiveness  of  the  contractor’s  implementation  of  internal  management 
controls. 

(Note:  The  EVMIG  will  mention  joint  government/contractor  surveillance  as  an  option 
and  will  reference  the  National  Defense  Industrial  Association  (NDIA)  Surveillance 
Guide.  Policy  will  be  developed  on  what  remedies  will  be  available  to  DoD  if 
surveillance  uncovers  that  a  contractor’s  EVMS  is  not  compliant  with  the  Industry 
Standard  (as  interpreted  by  the  Guidelines  Intent  document  for  complex  high-risk 
projects),  or  that  the  EV  data  being  reported  in  the  CPR  and  IMS  is  not  compliant  with 
what  is  required  by  the  respective  DIDs.) 

Recommendation  8:  An  Integrated  Baseline  Review  (IBR)  will  be  required  whenever 
EVM  is  required. 

IBRs  are  required  whenever  EVM  is  required  in  order  to  assess  that  the  contractor 
baseline  for  performing  the  work  is  achievable  and  that  both  the  contractor  and  customer 
understand  all  significant  risks.  An  IBR  is  simply  good  practice  at  the  start  of  any 
project  regardless  of  size,  and  it  can  be  tailored  as  appropriate  to  the  scope  of  the  work. 

Implementation  of  Policy  Changes 

It  is  not  proposed  that  the  new  EVM  policy  be  retroactive;  however,  the  changes  are 
aimed  at  improving  DoD  and  industry  EVM  practice,  so  there  will  be  no  prohibition  on 
programs  adopting  changes  and  negotiating  them  into  existing  contracts,  subcontracts,  or 
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other  agreements.  The  duration  remaining  on  a  contract  and  other  risk  factors  should  be 
considered  in  making  the  determination  to  modify  an  existing  contract  to  require  EVM. 
The  costs  associated  with  imposing  new  or  different  EVM  requirements  on  existing 
contracts  should  and  will  be  borne  by  the  government. 


91 


Appendix  C 


Project  Project  Project 

Project  Manager  Experience  Manager  a  Manager  b  Manager  c  Average 

Risk  Factor  Description  Range  Prob  Cost  Prob  Cost  Prob  Cost  AVE  PROB  AVE  COST  AVE  % 


0  to  1 5  years 

New 

80 

$ 

5,000 

20 

$ 

10,000 

85 

$ 

61.67 

$ 

5,000 

0.500% 

Age  of  Facility 

15  to  30  years 

Medium 

10 

$ 

40,000 

30 

$ 

25,000 

10 

$ 

70,000 

16.67 

$ 

45,000 

4.500% 

over  30  years 

Old 

10 

$ 

100,000 

50 

$ 

250,000 

5 

$ 

150,000 

21.67 

$ 

166,667 

16.667% 

Mild 

20 

$ 

25% 

$ 

25 

$ 

15.08 

$ 

0.000% 

Weather 

Average  weather 

Normal 

50 

$ 

10,000 

50% 

$ 

50 

$ 

50,000 

33.50 

$ 

20,000 

2.000% 

Severe 

30 

$ 

60,000 

25% 

$ 

30,000 

25 

$ 

100,000 

18.42 

$ 

63,333 

6.333% 

NONE 

15.00 

$0 

Asbestos 

Very  Little  encountered 

Low 

45 

$ 

10,000 

10% 

$ 

5,000 

50 

$ 

31.70 

$ 

5,000 

0.500% 

Moderate  amount 

Medium 

30 

$ 

60,000 

50% 

$ 

60,000 

25 

$ 

50,000 

18.50 

$ 

56,667 

5.667% 

High  level  encountered 

High 

25 

$ 

130,000 

40% 

$ 

100,000 

25 

$ 

150,000 

16.80 

$ 

126,667 

12.667% 

NONE 

15.00 

$0 

Sub-Surface  (Ground  and  interior  for 
renovation)  (Differing/unforseen  sight 
conditions) 

Little  or  No  unforseen 

Low 

40 

$ 

15,000 

50% 

$ 

60 

$ 

20,000 

33.50 

s 

11,667 

1.167% 

Moderate  amount 

Medium 

35 

$ 

55,000 

30 

$ 

45,000 

20 

$ 

70,000 

28.33 

$ 

56,667 

5.667% 

High  level  encountered 

High 

25 

$ 

150,000 

20 

$ 

100,000 

20 

$ 

150,000 

21.67 

$ 

133,333 

13.333% 

Little  experience 

Low 

20 

$ 

50,000 

5% 

$ 

20,000 

25 

$ 

70,000 

15.02 

s 

46,667 

4.667% 

Contractor  Experience 

Normal  experience 

Medium 

60 

$ 

15,000 

50% 

$ 

4,000 

50 

$ 

30,000 

36.83 

s 

16,333 

1.633% 

Extensive  experience 

High 

20 

$ 

45% 

$ 

25 

$ 

15.15 

$ 

0.000% 

Poor  performance 

Low 

25 

$ 

80,000 

10% 

$ 

100,000 

30 

$ 

100,000 

18.37 

$ 

93,333 

9.333% 

Contractor  Past  Performance 

Normal  performance 

Medium 

55 

$ 

20,000 

75% 

$ 

20,000 

60 

$ 

50,000 

38.58 

s 

30,000 

3.000% 

Exceptional  performance 

High 

20 

$ 

15% 

$ 

10 

$ 

10.05 

$ 

0.000% 

Contractor  Management 
Experience 

Little  experience 

Low 

30 

$ 

75,000 

15 

$ 

50,000 

10 

$ 

100,000 

18.33 

$ 

75,000 

7.500% 

Normal  experience 

Medium 

50 

$ 

20,000 

60 

$ 

15,000 

60 

$ 

50,000 

56.67 

$ 

28,333 

2.833% 

Extensive  experience 

High 

20 

$ 

25 

$ 

30 

$ 

25.00 

$ 

0.000% 

Little  experience 

Low 

15 

$ 

40,000 

10 

$ 

250,000 

30 

$ 

70,000 

18.33 

$ 

120,000 

12.000% 

Designer  Experience 

Normal  experience 

Medium 

60 

$ 

15,000 

60 

$ 

10,000 

60 

$ 

30,000 

60.00 

$ 

18,333 

1.833% 

Extensive  experience 

High 

25 

$ 

30 

$ 

10 

$ 

21.67 

$ 

0.000% 

Poor  performance 

Low 

15 

$ 

50,000 

30 

s 

200,000 

30 

$ 

100,000 

25.00 

s 

116,667 

11.667% 

Designer  Past  Performance 

Normal  performance 

Medium 

60 

$ 

20,000 

60 

$ 

40,000 

60 

$ 

50,000 

60.00 

$ 

36,667 

3.667% 

Exceptional  performance 

High 

25 

$ 

10 

$ 

10 

$ 

15.00 

$ 

0.000% 

Not  accurate 

Low 

60 

$ 

100,000 

50 

$ 

250,000 

60 

$ 

100,000 

56.67 

$ 

150,000 

15.000% 

Design  Accuracy 

Accurate 

Medium 

40 

$ 

35,000 

30 

$ 

10,000 

35 

$ 

40,000 

35.00 

s 

28,333 

2.833% 

Very  accurate 

High 

10 

$ 

20 

$ 

5 

$ 

11.67 

$ 

0.000% 

Design  Constructability  (Type 
of  Facility) 

Simple  Design 

Low 

15 

$ 

40 

$ 

30 

$ 

28.33 

$ 

0.000% 

Normal  Design 

Medium 

75 

$ 

30,000 

20 

$ 

60 

$ 

30,000 

51.67 

$ 

20,000 

2.000% 

Highly  Technical/Intricate 

High 

10 

$ 

85,000 

40 

$ 

25,000 

10 

$ 

150,000 

20.00 

$ 

86,667 

8.667% 

0-8  months 

Low 

5 

$ 

10,000 

20 

$ 

5,000 

15 

$ 

20,000 

13.33 

s 

11,667 

1.167% 

Project  Duration 

8-18  months 

Medium 

35 

$ 

30,000 

60 

$ 

15,000 

30 

$ 

50,000 

41.67 

$ 

31,667 

3.167% 

Over  1 8  months 

High 

60 

$ 

75,000 

20 

$ 

30,000 

55 

$ 

100,000 

45.00 

$ 

68,333 

6.833% 

NONE 

15.00 

$0 

Opportunity  for  Design  Changes 

Little  user  input 

Low 

15 

$ 

20,000 

20% 

$ 

20,000 

15 

$ 

100,000 

10.07 

s 

46,667 

4.667% 

Moderate  user  interaction 

Medium 

70 

$ 

65,000 

20% 

$ 

100,000 

60 

$ 

50,000 

43.40 

$ 

71,667 

7.167% 

Extensive  user  input 

High 

15 

$ 

100,000 

60% 

$ 

200,000 

25 

$ 

20,000 

13.53 

$ 

106,667 

10.667% 
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Appendix  D 


Sub  test() 

Dim  arraysize  As  Long 
Start  =  Timer 

Dim  temptree(l,  16)  As  Single 

'build  one  for  each  chance  node 

Dim  age(3,  2)  As  Double 

Dim  weather(3,  2)  As  Double 

Dim  asbestos(3,  2)  As  Double 

Dim  subsurface(3,  2)  As  Double 

Dim  contractorexperience(3,  2)  As  Double 

Dim  contractorpastperformance(3,  2)  As  Double 

Dim  contractormanagementexpertise(3,  2)  As  Double 

Dim  designaccuracy(3,  2)  As  Double 

Dim  constructability(3,  2)  As  Double 

Dim  duration(3, 2)  As  Double 

Dim  changes(3,  2)  As  Double 

Dim  designerexperience(3,  2)  As  Double 

Dim  designerpastperformance(3,  2)  As  Double 

Dim  designerproximity(3,  2)  As  Double 

Dim  Cumprob()  As  Single 

'constants 

cost  =  Sheets("CGF").Cells(2,  6) 

'build  for  each  level 

'first  number  is  level 

'second  number  is  l=prob  2  =  outcome 

'User  input 
agecount  =  0 
colnum  =  1 

While  Sheets("CGF").Cells(6,  colnum)  <>  "" 
agecount  =  agecount  +  1 
colnum  =  colnum  +  1 
Wend 

'User  input 
weathercount  =  0 
colnum  =  5 

While  Sheets("CGF").Cells(6,  colnum)  <>  "" 
weathercount  =  weathercount  +  1 
colnum  =  colnum  +  1 
Wend 

'User  input 
asbestoscount  =  0 
colnum  =  9 

While  Sheets("CGF").Cells(6,  colnum)  <>  "" 
asbestoscount  =  asbestoscount  +  1 
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colnum  =  colnum  +  1 
Wend 

'User  input 
sscount  =  0 
colnum  =  1 

While  Sheets("CGF").Cells(ll,  colnum)  <>  "" 
sscount  =  sscount  +  1 
colnum  =  colnum  +  1 
Wend 

'User  input 
cecount  =  0 
colnum  =  5 

While  Sheets("CGF").Cells(ll,  colnum)  <>  "" 
cecount  =  cecount  +  1 
colnum  =  colnum  +  1 
Wend 

'User  input 
cppcount  =  0 
colnum  =  9 

While  Sheets("CGF").Cells(ll,  colnum)  <>  "" 
cppcount  =  cppcount  +  1 
colnum  =  colnum  +  1 
Wend 


'User  input 
cmecount  =  0 
colnum  =  1 

While  Sheets("CGF").Cells(16,  colnum)  <>  "" 
cmecount  =  cmecount  +  1 
colnum  =  colnum  +  1 
Wend 

'User  input 
dacount  =  0 
colnum  =  5 

While  Sheets("CGF").Cells(16,  colnum)  <>  "" 
dacount  =  dacount  +  1 
colnum  =  colnum  +  1 
Wend 

'User  input 
cocount  =  0 
colnum  =  9 

While  Sheets("CGF").Cells(16,  colnum)  <>  "" 
cocount  =  cocount  +  1 
colnum  =  colnum  +  1 
Wend 
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'User  input 
ducount  =  0 
colnum  =  1 

While  Sheets("CGF").Cells(21,  colnum)  <>  "" 
ducount  =  ducount  +  1 
colnum  =  colnum  +  1 
Wend 

'User  input 
chcount  =  0 
colnum  =  5 

While  Sheets("CGF").Cells(21,  colnum)  <>  "" 
chcount  =  chcount  +  1 
colnum  =  colnum  +  1 
Wend 

'User  input 
decount  =  0 
colnum  =  9 

While  Sheets("CGF").Cells(21,  colnum)  <>  "" 
decount  =  decount  +  1 
colnum  =  colnum  +  1 
Wend 

'User  input 
dppcount  =  0 
colnum  =  1 

While  Sheets("CGF").Cells(26,  colnum)  <>  "" 
dppcount  =  dppcount  +  1 
colnum  =  colnum  +  1 
Wend 


age(l,  1)  =  Sheets("CGF").Cells(6,  1) 
age(l,  2)  =  Sheets("CGF").Cells(7, 1)  *  cost 
age(2, 1)  =  Sheets("CGF").Cells(6,  2) 
age(2,  2)  =  Sheets("CGF").Cells(7,  2)  *  cost 
age(3,  1)  =  Sheets("CGF").Cells(6,  3) 
age(3,  2)  =  Sheets("CGF").Cells(7,  3)  *  cost 

weather(l,  1)  =  Sheets("CGF").Cells(6,  5) 
weather(l,  2)  =  Sheets("CGF").Cells(7,  5)  *  cost 
weather(2, 1)  =  Sheets("CGF").Cells(6,  6) 
weather(2,  2)  =  Sheets("CGF").Cells(7,  6)  *  cost 
weather(3, 1)  =  Sheets("CGF").Cells(6,  7) 
weather(3,  2)  =  Sheets("CGF").Cells(7,  7)  *  cost 
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asbestos(l,  1)  =  Sheets("CGF").Cells(6,  9) 
asbestos(l,  2)  =  Sheets("CGF").Cells(7,  9)  *  cost 
asbestos(2, 1)  =  Sheets("CGF").Cells(6, 10) 
asbestos(2,  2)  =  Sheets("CGF").Cells(7, 10)  *  cost 
asbestos(3, 1)  =  Sheets("CGF").Cells(6, 11) 
asbestos(3,  2)  =  Sheets("CGF").Cells(7, 11)  *  cost 


subsurface(l,  1)  =  Sheets("CGF").Cells(ll,  1) 
subsurface(l,  2)  =  Sheets("CGF").Cells(12, 1)  *  cost 
subsurface(2,  1)  =  Sheets("CGF").Cells(ll,  2) 
subsurface(2,  2)  =  Sheets("CGF").Cells(12,  2)  *  cost 
subsurface(3, 1)  =  Sheets("CGF").Cells(ll,  3) 
subsurface(3,  2)  =  Sheets("CGF").Cells(12,  3)  *  cost 


contractorexperience(l,  1)  =  Sheets("CGF").Cells(ll,  5) 
contractorexperience(l,  2)  =  Sheets("CGF").Cells(12,  5)  *  cost 
contractorexperience(2, 1)  =  Sheets("CGF").Cells(ll,  6) 
contractorexperience(2,  2)  =  Sheets("CGF").Cells(12,  6)  *  cost 
contractorexperience(3, 1)  =  Sheets("CGF").Cells(ll,  7) 
contractorexperience(3,  2)  =  Sheets("CGF").Cells(12,  7)  *  cost 


contractorpastperformance(l,  1)  =  Sheets("CGF").Cells(ll,  9) 
contractorpastperformance(l,  2)  =  Sheets("CGF").Cells(12,  9)  *  cost 
contractorpastperformance(2, 1)  =  Sheets("CGF").Cells(ll,  10) 
contractorpastperformance(2,  2)  =  Sheets("CGF").Cells(12, 10)  *  cost 
contractorpastperformance(3, 1)  =  Sheets("CGF").Cells(ll,  11) 
contractorpastperformance(3,  2)  =  Sheets("CGF").Cells(12,  11)  *  cost 


contractormanagementexpertise(l,  1)  =  Sheets("CGF").Cells(16, 1) 
contractormanagementexpertise(l,  2)  =  Sheets("CGF").Cells(17, 1)  *  cost 
contractormanagementexpertise(2, 1)  =  Sheets("CGF").Cells(16,  2) 
contractormanagementexpertise(2,  2)  =  Sheets("CGF").Cells(17,  2)  *  cost 
contractormanagementexpertise(3,  1)  =  Sheets("CGF").Cells(16,  3) 
contractormanagementexpertise(3,  2)  =  Sheets("CGF").Cells(17,  3)  *  cost 


designaccuracy(l,  1)  =  Sheets("CGF").Cells(16,  5) 
designaccuracy(l,  2)  =  Sheets("CGF").Cells(17,  5)  *  cost 
designaccuracy(2, 1)  =  Sheets("CGF").Cells(16,  6) 
designaccuracy(2,  2)  =  Sheets("CGF").Cells(17,  6)  *  cost 
designaccuracy(3, 1)  =  Sheets("CGF").Cells(16,  7) 
designaccuracy(3,  2)  =  Sheets("CGF").Cells(17,  7)  *  cost 


constructability(l,  1)  =  Sheets("CGF").Cells(16,  9) 
constructability(l,  2)  =  Sheets("CGF").Cells(17,  9)  *  cost 
constructability(2, 1)  =  Sheets("CGF").Cells(16, 10) 
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constructability(2,  2)  =  Sheets("CGF").Cells(17,  10)  *  cost 
constructability(3, 1)  =  Sheets("CGF").Cells(16, 11) 
constructability(3,  2)  =  Sheets("CGF").Cells(17, 11)  *  cost 


duration(l,  1)  =  Sheets("CGF").Cells(21,  1) 
duration(l,  2)  =  Sheets("CGF").Cells(22, 1)  *  cost 
duration^,  1)  =  Sheets("CGF").Cells(21,  2) 
duration^,  2)  =  Sheets("CGF").Cells(22,  2)  *  cost 
duration^,  1)  =  Sheets("CGF").Cells(21,  3) 
duration(3,  2)  =  Sheets("CGF").Cells(22,  3)  *  cost 


changes(l,  1)  =  Sheets("CGF").Cells(21,  5) 
changes(l,  2)  =  Sheets("CGF").Cells(22,  5)  *  cost 
changes(2, 1)  =  Sheets("CGF").Cells(21,  6) 
changes(2,  2)  =  Sheets("CGF").Cells(22,  6)  *  cost 
changes(3, 1)  =  Sheets("CGF").Cells(21,  7) 
changes(3,  2)  =  Sheets("CGF").Cells(22,  7)  *  cost 

designerexperience(l,  1)  =  Sheets("CGF").Cells(21,  9) 
designerexperience(l,  2)  =  Sheets("CGF").Cells(22,  9)  *  cost 
designerexperience(2, 1)  =  Sheets("CGF").Cells(21,  10) 
designerexperience(2,  2)  =  Sheets("CGF").Cells(22,  10)  *  cost 
designerexperience(3, 1)  =  Sheets("CGF").Cells(21,  11) 
designerexperience(3,  2)  =  Sheets("CGF").Cells(22,  11)  *  cost 


designerpastperformance(l,  1)  =  Sheets("CGF").Cells(26, 1) 
designerpastperformance(l,  2)  =  Sheets("CGF").Cells(27, 1)  *  cost 
designerpastperformance(2, 1)  =  Sheets("CGF").Cells(26,  2) 
designerpastperformance(2,  2)  =  Sheets("CGF").Cells(27,  2)  *  cost 
designerpastperformance(3, 1)  =  Sheets("CGF").Cells(26,  3) 
designerpastperformance(3,  2)  =  Sheets("CGF").Cells(27,  3)  *  cost 


'designerproximity(l,  1)  =  Sheets("CGF").Cells(26,  5) 
'designerproximity(l,  2)  =  Sheets("CGF").Cells(27,  5)  *  cost 
'designerproximity(2,  1)  =  Sheets("CGF").Cells(26,  6) 
'designerproximity(2,  2)  =  Sheets("CGF").Cells(27,  6)  *  cost 
'designerproximity(3, 1)  =  Sheets("CGF").Cells(26,  7) 
'designerproximity(3,  2)  =  Sheets("CGF").Cells(27,  7)  *  cost 

ev  =  0 

For  i  =  1  To  agecount 

For  j  =  1  To  weathercount 
For  k  =  1  To  asbestoscount 
For  L  =  1  To  sscount 
For  m  =  1  To  cecount 
For  n  =  1  To  cppcount 
For  o  =  1  To  cmecount 
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For  p  =  1  To  dacount 
For  q  =  1  To  cocount 
For  R  =  1  To  ducount 
For  s  =  1  To  chcount 
For  t  =  1  To  decount 
For  u  =  1  To  dppcount 
'For  v  =  1  To  3 
Count  =  Count  +  1 

'build  one  statement  for  each  chance  node 
storetree(Count,  1)  =  i 
storetree(Count,  2)  =  j 
storetree(Count,  3)  =  k 
storetree(Count,  4)  =  L 
storetree(Count,  5)  =  m 
storetree(Count,  6)  =  n 
storetree(Count,  7)  =  o 
storetree(Count,  8)  =  p 
storetree(Count,  9)  =  q 
storetree(Count,  10)  =  R 
storetree(Count,  11)  =  s 
storetree(Count,  12)  =  t 
storetree(Count,  13)  =  u 
storetree(Count,  14)  =  0 
'build  one  equation  for  prob  and  outcome 
Prob  =  age(i,  1)  *  weather(j,  1)  *  asbestos(k,  1)  * 
subsurface(L,  1)  *  contractorexperience(m,  1)  *  contractorpastperformance(n,  1)  * 
contractormanagementexpertise(o,  1)  *  designaccuracy(p,  1)  *  constructability(q,  1)  * 
duration(R,  1)  *  changes(s,  1)  *  designerexperience(t,  1)  *  designerpastperformance(u,  1) 

outcome  =  age(i,  2)  +  weather(j,  2)  +  asbestos(k,  2)  + 
subsurface(L,  2)  +  contractorexperience(m,  2)  +  contractorpastperformance(n,  2)  + 
contractormanagementexpertise(o,  2)  +  designaccuracy(p,  2)  +  constructability(q,  2)  + 
duration(R,  2)  +  changes(s,  2)  +  designerexperience(t,  2)  +  designerpastperformance(u,  2) 

ev  =  ev  +  Prob  *  outcome 
storetree(Count,  15)  =  Prob 
storetree(Count,  16)  =  outcome 
'Next 
Next 
Next 
Next 
Next 
Next 
Next 
Next 
Next 
Next 
Next 
Next 
Next 
Next 

arraysize  =  Count 
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Sheets(2).Cells(l,  4)  =  ev 

Finish  =  Timer  '  Set  end  time, 
totaltime  =  (Finish  -  Start)  /  60 

Sheets(2).Cells(l,  6)  =  "It  took  "  &  Format(totaltime,  "######.##")  &  "  minutes  to  run" 
'sort  storetree 

QuickSortL  1,  arraysize,  cost 

'Sheets("sheet2").Range("Al:D6200").Select 

Selection.ClearContents 

Count  =  1 

total =  1 

Cumprobtotal  =  0 
While  Count  <=  arraysize 

ReDim  Preserve  Cumprob(3,  total) 
temp  =  storetree(Count,  16) 

Prob  =  0 

While  storetree(Count,  16)  =  temp 

Cumprobtotal  =  Cumprobtotal  +  storetree(Count,  15) 

Prob  =  Prob  +  storetree(Count,  15) 

Count  =  Count  +  1 
Wend 

Cumprob(l,  total)  =  temp 
Cumprob(2,  total)  =  Prob 
Cumprob(3,  total)  =  Cumprobtotal 
total  =  total  +  1 
Wend 

Sheets(2).Cells(l,  4)  =  ev 

For  i  =  1  To  total  -  1 

Sheets(2).Cells(i,  1)  =  Cumprob(l,  i) 

Sheets(2).Cells(i,  2)  =  Cumprob(2,  i) 

Sheets(2).Cells(i,  3)  =  Cumprob(3,  i) 

Next  i 

End  Sub 


Sub  QuickSortL(L  As  Long,  R  As  Long,  cost) 
storetree(L  -  1, 16)  =  0 
storetree(R  +  1,16)  =  10000  *  cost 
QuickL  L,  R 
InsertL  L,  R 
End  Sub 

Sub  QuickL(L  As  Long,  R  As  Long) 

Dim  MED  As  Long 
Dim  LP  As  Long 
Dim  RP  As  Long 
Dim  Pivot  As  Long 
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Dim  TMP  As  Double 


IfR-L>  12  Then 
MED  =  (L  +  R)  \  2 
For  i  =  1  To  16 

TMP  =  storetree(MED,  i) 
storetree(MED,  i)  =  storetree(L,  i) 
storetree(L,  i)  =  TMP 
Next  i 

If  storetree(L  +  1, 16)  >  storetree(R,  16)  Then 
For  i  =  1  To  16 

TMP  =  storetree(L  +  1,  i) 
storetree(L  +  1,  i)  =  storetree(R,  i) 
storetree(R,  i)  =  TMP 
Next  i 
End  If 

If  storetree(L,  16)  >  storetree(R,  16)  Then 
For  i  =  1  To  16 

TMP  =  storetree(L,  i) 
storetree(L,  i)  =  storetree(R,  i) 
storetree(R,  i)  =  TMP 
Next  i 
End  If 

If  storetree(L  +  1, 16)  >  storetree(L,  16)  Then 
For  i  =  1  To  16 

TMP  =  storetree(L  +  1,  i) 
storetree(L  +  1,  i)  =  storetree(L,  i) 
storetree(L,  i)  =  TMP 
Next  i 
End  If 

Pivot  =  storetree(L,  16) 

LP  =  L 
RP  =  R  +  1 
Do 
Do 

LP  =  LP  +  1 

Loop  While  storetree(LP,  16)  <  Pivot 
Do 

RP  =  RP  -  1 

Loop  While  storetree(RP,  16)  >  Pivot 
If  RP  <=  LP  Then  Exit  Do 
For  i  =  1  To  16 

TMP  =  storetree(LP,  i) 
storetree(LP,  i)  =  storetree(RP,  i) 
storetree(RP,  i)  =  TMP 
Next  i 
Loop 

For  i  =  1  To  16 

TMP  =  storetree(L,  i) 
storetree(L,  i)  =  storetree(RP,  i) 
storetree(RP,  i)  =  TMP 
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Next  i 

If  (RP  -  1)  -  L  <  R  -  LP  Then 
QuickL  L,  RP  -  1 
QuickL  LP,  R 
Else 

QuickL  LP,  R 
QuickL  L,  RP  -  1 
End  If 
End  If 
End  Sub 

Sub  InsertL(L  As  Long,  R  As  Long) 

Dim  LP  As  Long 
Dim  RP  As  Long 
Dim  TMP(1, 16)  As  Double 

For  RP  =  L  +  1  To  R 
For  i  =  1  To  16 

TMP(1,  i)  =  storetree(RP,  i) 

Next  i 

For  LP  =  RP  To  L  +  1  Step  -1 

If  TMP(1,  16)  <  storetree(LP  -  1, 16)  Then 
For  i  =  1  To  16 

storetree(LP,  i)  =  storetree(LP  - 1,  i) 
Next  i 
Else 

Exit  For 
End  If 
Next  LP 
For  i  =  1  To  16 

storetree(LP,  i)  =  TMP(1,  i) 

Next  i 
Next  RP 
End  Sub 
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Appendix  E 


Dim  storetree(1600000, 16)  As  Single 
Sub  test() 

Dim  arraysize  As  Long 
Start  =  Timer 

Dim  temptree(l,  16)  As  Single 

'build  one  for  each  chance  node 

Dim  age(3,  2)  As  Double 

Dim  weather(3,  2)  As  Double 

Dim  asbestos(3,  2)  As  Double 

Dim  subsurface(3,  2)  As  Double 

Dim  contractorexperience(3,  2)  As  Double 

Dim  contractorpastperformance(3,  2)  As  Double 

Dim  contractormanagementexpertise(3,  2)  As  Double 

Dim  designaccuracy(3,  2)  As  Double 

Dim  constructability(3,  2)  As  Double 

Dim  duration(3, 2)  As  Double 

Dim  changes(3,  2)  As  Double 

Dim  designerexperience(3,  2)  As  Double 

Dim  designerpastperformance(3,  2)  As  Double 

Dim  Cumprob()  As  Single 

'constants 

cost  =  Sheets("CGF").Cells(2, 10) 

'build  for  each  level 

'first  number  is  level 

'second  number  is  l=prob  2  =  outcome 

'User  input  for  Age 
If  Sheets("CGF").Cells(6,  4)  =  0  Then 
age(l,  1)  =  1 
age(l,  2)  =  0 
agecount  =  1 
End  If 

If  Sheets("CGF").Cells(6,  4)  >  0  And  Sheets("CGF").Cells(6,  4)  <  15  Then 
age(l,  1)  =  1 
age(l,  2)  =  0.005  *  cost 
agecount  =  1 
End  If 

If  Sheets("CGF").Cells(6,  4)  >=  15  And  Sheets("CGF").Cells(6,  4)  <  30  Then 
age(l,  1)  =  1 
age(l,  2)  =  0.045  *  cost 
agecount  =  1 
End  If 

If  Sheets("CGF").Cells(6,  4)  >=  30  Then 
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age(l,  1)  =  1 

age(l,  2)  =  0.16667  *  cost 
agecount  =  1 
End  If 

'User  input  for  Duration 
If  Sheets("CGF").Cells(21,  4)  =  0  Then 
duration(l,  1)  =  1 
duration(l,  2)  =  0 
ducount  =  1 
End  If 

If  Sheets("CGF").Cells(21,  4)  >  0  And  Sheets("CGF").Cells(21,  4)  <  8  Then 
duration(l,  1)  =  1 
duration(l,  2)  =  0.005  *  cost 
ducount  =  1 
End  If 

If  Sheets("CGF").Cells(21,  4)  >=  8  And  Sheets("CGF").Cells(21,  4)  <  18  Then 
duration(l,  1)  =  1 
duration(l,  2)  =  0.045  *  cost 
ducount  =  1 
End  If 

If  Sheets("CGF").Cells(21,  4)  >=  18  Then 
duration(l,  1)  =  1 
duration(l,  2)  =  0.16667  *  cost 
ducount  =  1 
End  If 


'User  input 
weathercount  =  0 
colnum  =  9 

While  Sheets("CGF").Cells(6,  colnum)  <>  "" 
weathercount  =  weathercount  +  1 
colnum  =  colnum  +  1 
Wend 

'User  input 
asbestoscount  =  0 
colnum  =  15 

While  Sheets("CGF").Cells(6,  colnum)  <>  "" 
asbestoscount  =  asbestoscount  +  1 
colnum  =  colnum  +  1 
Wend 

'User  input 
sscount  =  0 
colnum  =  3 
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While  Sheets("CGF").Cells(ll,  colnum)  <>  "" 
sscount  =  sscount  +  1 
colnum  =  colnum  +  1 
Wend 

'User  input 
cecount  =  0 
colnum  =  9 

While  Sheets("CGF").Cells(ll,  colnum)  <>  "" 
cecount  =  cecount  +  1 
colnum  =  colnum  +  1 
Wend 

'User  input 
cppcount  =  0 
colnum  =  15 

While  Sheets("CGF").Cells(ll,  colnum)  <>  "" 
cppcount  =  cppcount  +  1 
colnum  =  colnum  +  1 
Wend 


'User  input 
cmecount  =  0 
colnum  =  3 

While  Sheets("CGF").Cells(16,  colnum)  <>  "" 
cmecount  =  cmecount  +  1 
colnum  =  colnum  +  1 
Wend 

'User  input 
dacount  =  0 
colnum  =  9 

While  Sheets("CGF").Cells(16,  colnum)  <>  "" 
dacount  =  dacount  +  1 
colnum  =  colnum  +  1 
Wend 

'User  input 
cocount  =  0 
colnum  =  15 

While  Sheets("CGF").Cells(16,  colnum)  <>  "" 
cocount  =  cocount  +  1 
colnum  =  colnum  +  1 
Wend 


'User  input 
chcount  =  0 
colnum  =  9 

While  Sheets("CGF").Cells(21,  colnum)  <>  "" 
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chcount  =  chcount  +  1 
colnum  =  colnum  +  1 
Wend 

'User  input 
decount  =  0 
colnum  =  15 

While  Sheets("CGF").Cells(21,  colnum)  <>  "" 
decount  =  decount  +  1 
colnum  =  colnum  +  1 
Wend 

'User  input 
dppcount  =  0 
colnum  =  3 

While  Sheets("CGF").Cells(26,  colnum)  <>  "" 
dppcount  =  dppcount  +  1 
colnum  =  colnum  +  1 
Wend 


weather(l,  1)  =  Sheets("CGF").Cells(6,  9) 
weather(l,  2)  =  Sheets("CGF").Cells(7,  9)  *  cost 
weather(2, 1)  =  Sheets("CGF").Cells(6, 10) 
weather(2,  2)  =  Sheets("CGF").Cells(7, 10)  *  cost 
weather(3, 1)  =  Sheets("CGF").Cells(6, 11) 
weather(3,  2)  =  Sheets("CGF").Cells(7, 11)  *  cost 


asbestos(l,  1)  =  Sheets("CGF").Cells(6, 15) 
asbestos(l,  2)  =  Sheets("CGF").Cells(7, 15)  *  cost 
asbestos(2, 1)  =  Sheets("CGF").Cells(6, 16) 
asbestos(2,  2)  =  Sheets("CGF").Cells(7, 16)  *  cost 
asbestos(3, 1)  =  Sheets("CGF").Cells(6, 17) 
asbestos(3,  2)  =  Sheets("CGF").Cells(7, 17)  *  cost 

subsurface(l,  1)  =  Sheets("CGF").Cells(ll,  3) 
subsurface(l,  2)  =  Sheets("CGF").Cells(12,  3)  *  cost 
subsurface(2, 1)  =  Sheets("CGF").Cells(ll,  4) 
subsurface(2,  2)  =  Sheets("CGF").Cells(12,  4)  *  cost 
subsurface(3,  1)  =  Sheets("CGF").Cells(ll,  5) 
subsurface(3,  2)  =  Sheets("CGF").Cells(12,  5)  *  cost 

contractorexperience(l,  1)  =  Sheets("CGF").Cells(ll,  9) 
contractorexperience(l,  2)  =  Sheets("CGF").Cells(12,  9)  *  cost 
contractorexperience(2, 1)  =  Sheets("CGF").Cells(ll,  10) 
contractorexperience(2,  2)  =  Sheets("CGF").Cells(12, 10)  *  cost 
contractorexperience(3, 1)  =  Sheets("CGF").Cells(ll,  11) 
contractorexperience(3,  2)  =  Sheets("CGF").Cells(12, 11)  *  cost 
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contractorpastperformance(l,  1)  =  Sheets("CGF").Cells(ll,  15) 
contractorpastperformance(l,  2)  =  Sheets("CGF").Cells(12, 15)  *  cost 
contractorpastperformance(2,  1)  =  Sheets("CGF").Cells(ll,  16) 
contractorpastperformance(2,  2)  =  Sheets("CGF").Cells(12, 16)  *  cost 
contractorpastperformance(3, 1)  =  Sheets("CGF").Cells(ll,  17) 
contractorpastperformance(3,  2)  =  Sheets("CGF").Cells(12, 17)  *  cost 


contractormanagementexpertise(l,  1)  =  Sheets("CGF").Cells(16,  3) 
contractormanagementexpertise(l,  2)  =  Sheets("CGF").Cells(17,  3)  *  cost 
contractormanagementexpertise(2, 1)  =  Sheets("CGF").Cells(16,  4) 
contractormanagementexpertise(2,  2)  =  Sheets("CGF").Cells(17,  4)  *  cost 
contractormanagementexpertise(3, 1)  =  Sheets("CGF").Cells(16,  5) 
contractormanagementexpertise(3,  2)  =  Sheets("CGF").Cells(17,  5)  *  cost 


designaccuracy(l,  1)  =  Sheets("CGF").Cells(16,  9) 
designaccuracy(l,  2)  =  Sheets("CGF").Cells(17,  9)  *  cost 
designaccuracy(2,  1)  =  Sheets("CGF").Cells(16, 10) 
designaccuracy(2,  2)  =  Sheets("CGF").Cells(17,  10)  *  cost 
designaccuracy(3, 1)  =  Sheets("CGF").Cells(16, 11) 
designaccuracy(3,  2)  =  Sheets("CGF").Cells(17, 11)  *  cost 


constructability(l,  1)  =  Sheets("CGF").Cells(16, 15) 
constructability(l,  2)  =  Sheets("CGF").Cells(17, 15)  *  cost 
constructability(2,  1)  =  Sheets("CGF").Cells(16, 16) 
constructability(2,  2)  =  Sheets("CGF").Cells(17,  16)  *  cost 
constructability(3, 1)  =  Sheets("CGF").Cells(16, 17) 
constructability(3,  2)  =  Sheets("CGF").Cells(17, 17)  *  cost 


changes(l,  1)  =  Sheets("CGF").Cells(21,  9) 
changes(l,  2)  =  Sheets("CGF").Cells(22,  9)  *  cost 
changes(2, 1)  =  Sheets("CGF").Cells(21, 10) 
changes(2,  2)  =  Sheets("CGF").Cells(22, 10)  *  cost 
changes(3, 1)  =  Sheets("CGF").Cells(21, 11) 
changes(3,  2)  =  Sheets("CGF").Cells(22, 11)  *  cost 

designerexperience(l,  1)  =  Sheets("CGF").Cells(21, 15) 
designerexperience(l,  2)  =  Sheets("CGF").Cells(22,  15)  *  cost 
designerexperience(2, 1)  =  Sheets("CGF").Cells(21, 16) 
designerexperience(2,  2)  =  Sheets("CGF").Cells(22,  16)  *  cost 
designerexperience(3, 1)  =  Sheets("CGF").Cells(21, 17) 
designerexperience(3,  2)  =  Sheets("CGF").Cells(22, 17)  *  cost 


designerpastperformance(l,  1)  =  Sheets("CGF").Cells(26,  3) 
designerpastperformance(l,  2)  =  Sheets("CGF").Cells(27,  3)  *  cost 
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designerpastperformance(2, 1)  =  Sheets("CGF").Cells(26,  4) 
designerpastperformance(2,  2)  =  Sheets("CGF").Cells(27,  4)  *  cost 
designerpastperformance(3, 1)  =  Sheets("CGF").Cells(26,  5) 
designerpastperformance(3,  2)  =  Sheets("CGF").Cells(27,  5)  *  cost 


ev  =  0 

For  i  =  1  To  agecount 

For  j  =  1  To  weathercount 
For  k  =  1  To  asbestoscount 
For  L  =  1  To  sscount 
For  m  =  1  To  cecount 
For  n  =  1  To  cppcount 
For  o  =  1  To  cmecount 
For  p  =  1  To  dacount 
For  q  =  1  To  cocount 
For  R  =  1  To  ducount 
For  s  =  1  To  chcount 
For  t  =  1  To  decount 
For  u  =  1  To  dppcount 
'For  v  =  1  To  3 
Count  =  Count  +  1 

'build  one  statement  for  each  chance  node 
storetree(Count,  1)  =  i 
storetree(Count,  2)  =  j 
storetree(Count,  3)  =  k 
storetree(Count,  4)  =  L 
storetree(Count,  5)  =  m 
storetree(Count,  6)  =  n 
storetree(Count,  7)  =  o 
storetree(Count,  8)  =  p 
storetree(Count,  9)  =  q 
storetree(Count,  10)  =  R 
storetree(Count,  11)  =  s 
storetree(Count,  12)  =  t 
storetree(Count,  13)  =  u 
storetree(Count,  14)  =  0 
'build  one  equation  for  prob  and  outcome 
Prob  =  age(i,  1)  *  weather(j,  1)  *  asbestos(k,  1)  * 
subsurface(L,  1)  *  contractorexperience(m,  1)  *  contractorpastperformance(n,  1)  * 
contractormanagementexpertise(o,  1)  *  designaccuracy(p,  1)  *  constructability(q,  1)  * 
duration(R,  1)  *  changes(s,  1)  *  designerexperience(t,  1)  *  designerpastperformance(u,  1) 

outcome  =  age(i,  2)  +  weather(j,  2)  +  asbestos(k,  2)  + 
subsurface(L,  2)  +  contractorexperience(m,  2)  +  contractorpastperformance(n,  2)  + 
contractormanagementexpertise(o,  2)  +  designaccuracy(p,  2)  +  constructability(q,  2)  + 
duration(R,  2)  +  changes(s,  2)  +  designerexperience(t,  2)  +  designerpastperformance(u,  2) 

ev  =  ev  +  Prob  *  outcome 
storetree(Count,  15)  =  Prob 
storetree(Count,  16)  =  outcome 
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'Next 

Next 

Next 

Next 

Next 

Next 

Next 

Next 

Next 

Next 

Next 

Next 

Next 

Next 

arraysize  =  Count 
Sheets(2).Cells(l,  4)  =  ev 

Finish  =  Timer  '  Set  end  time, 
totaltime  =  (Finish  -  Start)  /  60 

Sheets(2).Cells(l,  6)  =  "It  took  "  &  Format(totaltime,  "######.##")  &  "  minutes  to  run" 
'sort  storetree 

QuickSortL  1,  arraysize,  cost 

Sheets(2).Range("Al:D62000").Select 

Selection.ClearContents 

Count  =  1 

total =  1 

Cumprobtotal  =  0 
While  Count  <=  arraysize 

ReDim  Preserve  Cumprob(3,  total) 
temp  =  storetree(Count,  16) 

Prob  =  0 

While  storetree(Count,  16)  =  temp 

Cumprobtotal  =  Cumprobtotal  +  storetree(Count,  15) 

Prob  =  Prob  +  storetree(Count,  15) 

Count  =  Count  +  1 
Wend 

Cumprob(l,  total)  =  temp 
Cumprob(2,  total)  =  Prob 
Cumprob(3,  total)  =  Cumprobtotal 
total  =  total  +  1 
Wend 

Sheets(2).Cells(l,  4)  =  ev 

For  i  =  1  To  total  -  1 

Sheets(2).Cells(i,  1)  =  Cumprob(l,  i) 

Sheets(2).Cells(i,  2)  =  Cumprob(2,  i) 

Sheets(2).Cells(i,  3)  =  Cumprob(3,  i) 

Next  i 
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End  Sub 


Sub  QuickSortL(L  As  Long,  R  As  Long,  cost) 
storetree(L  -  1,  16)  =  0 
storetree(R  +  1,16)  =  10000  *  cost 
QuickL  L,  R 
InsertL  L,  R 
End  Sub 

Sub  QuickL(L  As  Long,  R  As  Long) 

Dim  MED  As  Long 
Dim  LP  As  Long 
Dim  RP  As  Long 
Dim  Pivot  As  Long 
Dim  TMP  As  Double 

IfR-L>  12  Then 
MED  =  (L  +  R)  \  2 
For  i  =  1  To  16 

TMP  =  storetree(MED,  i) 
storetree(MED,  i)  =  storetree(L,  i) 
storetree(L,  i)  =  TMP 
Next  i 

If  storetree(L  +  1, 16)  >  storetree(R,  16)  Then 
For  i  =  1  To  16 

TMP  =  storetree(L  +  1,  i) 
storetree(L  +  1,  i)  =  storetree(R,  i) 
storetree(R,  i)  =  TMP 
Next  i 
End  If 

If  storetree(L,  16)  >  storetree(R,  16)  Then 
For  i  =  1  To  16 

TMP  =  storetree(L,  i) 
storetree(L,  i)  =  storetree(R,  i) 
storetree(R,  i)  =  TMP 
Next  i 
End  If 

If  storetree(L  +  1,  16)  >  storetree(L,  16)  Then 
For  i  =  1  To  16 

TMP  =  storetree(L  +  1,  i) 
storetree(L  +  1,  i)  =  storetree(L,  i) 
storetree(L,  i)  =  TMP 
Next  i 
End  If 

Pivot  =  storetree(L,  16) 

LP  =  L 
RP  =  R  +  1 
Do 
Do 

LP  =  LP  +  1 
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Loop  While  storetree(LP,  16)  <  Pivot 
Do 

RP  =  RP  -  1 

Loop  While  storetree(RP,  16)  >  Pivot 
If  RP  <=  LP  Then  Exit  Do 
For  i  =  1  To  16 

TMP  =  storetree(LP,  i) 
storetree(LP,  i)  =  storetree(RP,  i) 
storetree(RP,  i)  =  TMP 
Next  i 
Loop 

For  i  =  1  To  16 

TMP  =  storetree(L,  i) 
storetree(L,  i)  =  storetree(RP,  i) 
storetree(RP,  i)  =  TMP 
Next  i 

If  (RP  -  1)  -  L  <  R  -  LP  Then 
QuickL  L,  RP  -  1 
QuickL  LP,  R 
Else 

QuickL  LP,  R 
QuickL  L,  RP  -  1 
End  If 
End  If 
End  Sub 

Sub  InsertL(L  As  Long,  R  As  Long) 

Dim  LP  As  Long 
Dim  RP  As  Long 
Dim  TMP(1, 16)  As  Double 

For  RP  =  L  +  1  To  R 
For  i  =  1  To  16 

TMP(1,  i)  =  storetree(RP,  i) 

Next  i 

For  LP  =  RP  To  L  +  1  Step  -1 

If  TMP(1, 16)  <  storetree(LP  -  1, 16)  Then 
For  i  =  1  To  16 

storetree(LP,  i)  =  storetree(LP  -  1,  i) 
Next  i 
Else 

Exit  For 
End  If 
Next  LP 
For  i  =  1  To  16 

storetree(LP,  i)  =  TMP(1,  i) 

Next  i 
Next  RP 
End  Sub 
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